[SCIP] [EXTERNAL] Re: Performance problem using SCIP with OR-Tools vs. SCIP shell

Mon Aug 15 18:27:08 CEST 2022

Hello Ambrose,

Thanks for looking into this and for your helpful feedback. I will try your suggested setting experimentations and OR-Tools team is also looking this further.

  *   Rich Levinson
  *   NASA Ames Research Center

________________________________
From: Scip <scip-bounces at zib.de> on behalf of Ambros Gleixner <gleixner at zib.de>
Sent: Thursday, August 11, 2022 8:24 PM
To: scip at zib.de <scip at zib.de>
Subject: [EXTERNAL] Re: [SCIP] Performance problem using SCIP with OR-Tools vs. SCIP shell

Dear Rich,

I am not familiar with how SCIP is included in OR-Tools, but by quickly
looking at the logs the difference in performance could indeed be
related to the underlying LP solver.

You can see the output line "(node 1) unresolved numerical troubles in
LP 7 -- using pseudo solution instead (loop 1)", which indicates that
SCIP in OR-Tools is not able to process the root node properly.  One can
also see that the dualbound is massively worse on the OR-Tools runs and
does not improve during tree search.

As a result, the OR-Tools runs mainly look like enumeration to me (or
branch-and-propagate) rather than branch-and-bound.

(The 200s vs. 600s degradation is most likely only performance
variability, I would not read anything into this.)

Hope that helps a bit at least.  You might want to experiment with the
parameters regarding LP numerics such as

numerics/dualfeastol
lp/checkdualfeas
lp/checkprimfeas
lp/checkstability

but it might be that only exchanging the LP solver will help.

Best,
Ambros

Am 11.08.22 um 21:14 schrieb Levinson, Richard J. (ARC-TI)[KBR Wyle
Services, LLC]:
> Hello,
>
> We are using Google OR-Tools with SCIP solver and see huge differences
> in solve times, using exactly the same lp model file, when we call SCIP
> directly vs. within OR-Tools.
>
> We create the model inside OR-Tools, and save the lp file so we can load
> it into SCIP's shell.
>
> Using OR-Tools/SCIP, solve times are (~200 secs to ~700 secs). It took
> 200 secs until we updated to the latest OR-Tools (to ensure OR-Tools was
> using SCIP 8), and now it takes 700 secs (the update more than doubled
> the solve time).
>
> However, if we load that same model into the SCIP shell, or call ./scip
> -f and pass in the file, then it only takes ~ 7 seconds to solve (100
> times faster than solving it via OR-tools).
>
> Our application requires solve times < 30 seconds. We'd appreciate any
> suggestions about why this is happening and what we can do to reduce our
> solve time within OR-Tools so it's closer to the 7 seconds we get with
> SCIP shell or command line?
>
> My guess is perhaps the SCIP parameter settings are different between
> the two cases, but not sure how to load a SCIP settings file into
> OR-Tools. Is that possible?  Originally I thought it might be that
> OR-Tools and the SCIP shell are using different versions of SCIP, but
> since I updated OR-Tools, I think both cases use SCIP 8+. OR-Tool
> support says they are using a post v8 because it contains a bug fix.
>
> OR-Tool support tells me that SCIP shell uses SOPLEX but the embedded
> SCIP uses GLOP, but he thought that would only account for 10%
> difference at most. Is this a possible cause of the problem?
>
> I'm attaching 5 files, the lp file and the solver log for 2 fast cases
> (SCIP without OR-Tools) and 2 slow cases (OR-Tools calling SCIP):
>
> All cases produce the same optimal objective, but the fast and slow
> cases show very different solver behaviors. The fast cases do a restart
> quickly after pre-solving, but the slow cases don't.
>
> Both fast cases have only 91 nodes, while the 200 sec case has 25,000
> and the 600 sec case has 117,000 nodes. I am hoping someone can look at
> the log files and figure out what settings would be appropriate to get
> OR-Tools behavior to match the SCIP standalone behavior.
>
>  1. *mplan.lp: *The model file created within OR-Tools but then saved
>     out and loaded into the shell/command line. When we load this into
>     SCIP shell, it takes 7 seconds to solve.
>  2. *scipShell.txt *(fast): The solver progress log when we load
>     mplan.lp into the SCIP shell, which takes only 7 seconds to solve.
>  3. *scipCmdLine-f.txt *(fast):  The solver progress log when we call
>     scip from command line with -f option to supply mplan.lp. This also
>     takes about 7 seconds to solve.
>  4. *orToolsPythonCmdLine_200s.txt *(slow):  The solver progress log
>     when we call your Python code (which calls OR-Tools) from a command
>     line. This was before I updated to the latest OR-tools.
>  5. *orToolsPythonCmdLine_660s.txt (*slow): Same as
>     orToolsPythonCmdLine_200s.txt above, except this is after updating
>     to the latest OR-Tools. This caused the solve times to more than
>     double from 200s to 670s.
>
> Any suggestions are appreciated.
>
> Thank you,
>        Rich Levinson
>        NASA Ames Research Center.
>
> _______________________________________________
> Scip mailing list
> Scip at zib.de
> https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Flistserv.zib.de%2Fmailman%2Flistinfo%2Fscip&data=05%7C01%7Crichard.j.levinson%40nasa.gov%7Cb76ae1b4ba494b02f23408da7c12c1c3%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637958717246645392%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=cp1BLFsiDHw8Muvj6hRaUOomEu4S9MTziKlKl4JrKsY%3D&reserved=0
_______________________________________________
Scip mailing list
Scip at zib.de
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Flistserv.zib.de%2Fmailman%2Flistinfo%2Fscip&data=05%7C01%7Crichard.j.levinson%40nasa.gov%7Cb76ae1b4ba494b02f23408da7c12c1c3%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637958717246645392%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=cp1BLFsiDHw8Muvj6hRaUOomEu4S9MTziKlKl4JrKsY%3D&reserved=0
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listserv.zib.de/pipermail/scip/attachments/20220815/41b9db53/attachment.html>