[phenixbb] temp files

[James Holton]

Thank you Tom!

Ok.  So, in order to change the default I need to go through the code 
looking for "temp_dir" and change things?


On 4/24/2021 10:16 AM, Tom Terwilliger wrote:
> Hi James,
>
> There is no overall Phenix temp directory specification, but most of 
> the temp_dir usage is from autosol/autobuild/ligandfit/map_to_model.  
> Each of these has the keyword "temp_dir=xxxx" which you should be
> able to set to any directory you want (and local is better as you 
> note).  Most programs using a temp_dir also have a keyword 
> clean_up=True as well.
>
> All the best,
> Tom T
>
> On Sat, Apr 24, 2021 at 10:51 AM James Holton <jmholton at lbl.gov 
> <mailto:jmholton at lbl.gov>> wrote:
>
>     Thank you Li-Wei
>
>     Definitely not placing blame on one program. Phenix.autobuild is
>     another
>     big temp file producer.  So is XDS. Clearly this ligand run was a
>     case
>     of a misconfigured, runaway task that never finished. However, the
>     files
>     lingered on disk, eating up inodes for 3 years!
>
>     The reason I'm asking is I think there are significant performance
>     increases to be gained by using fast, local storage for scratch
>     files.
>     This is not just in speed but storage and overall system/cluster
>     performance.  Very few things are more expensive than an NFS write!
>
>     Does anyone know how to change the default temp file location across
>     phenix ?  Is this a cctbx thing?
>
>     Thanks
>
>     -James
>
>
>     On 4/23/2021 9:38 PM, Li-Wei Hung wrote:
>     > Hi James,
>     >
>     > I'll leave the global Phenix temp aspect to Billy.
>     > For ligand identification specifically, the working directory is
>     where
>     > all the files are located.  The program will purge most of the
>     > intermediate files upon completion. If the user interrupted the
>     runs
>     > or if the program crashed at certain spots, the purge mechanism
>     might
>     > not kick in. Even so, it'd take many runs to accumulate 20e6 (2e7?)
>     > files. In any case, you've got a point and I'll look into salvaging
>     > intermediate files of ligand identification as soon as they are not
>     > needed in the process.
>     >
>     > Thanks,
>     >
>     > Li-Wei
>     >
>     > On 4/23/2021 7:03 PM, James Holton wrote:
>     >> Hello all,
>     >>
>     >> Is there a way to configure phenix at install time (or perhaps
>     >> post-install) to put temporary files under /tmp ?  I just had to
>     >> delete 20e6 temp files over NFS from a single user's phenix ligand
>     >> identification run.  The delete took almost a month.
>     >>
>     >> Apologies if I am neglecting to look somewhere obvious in the
>     >> documentation,
>     >>
>     >> Happy Weekend!
>     >>
>     >> -James Holton
>     >> MAD Scientist
>     >>
>     >> _______________________________________________
>     >> phenixbb mailing list
>     >> phenixbb at phenix-online.org <mailto:phenixbb at phenix-online.org>
>     >> http://phenix-online.org/mailman/listinfo/phenixbb
>     <http://phenix-online.org/mailman/listinfo/phenixbb>
>     >> Unsubscribe: phenixbb-leave at phenix-online.org
>     <mailto:phenixbb-leave at phenix-online.org>
>     >
>
>     _______________________________________________
>     phenixbb mailing list
>     phenixbb at phenix-online.org <mailto:phenixbb at phenix-online.org>
>     http://phenix-online.org/mailman/listinfo/phenixbb
>     <http://phenix-online.org/mailman/listinfo/phenixbb>
>     Unsubscribe: phenixbb-leave at phenix-online.org
>     <mailto:phenixbb-leave at phenix-online.org>
>
>
>
> -- 
> Thomas C Terwilliger
> Laboratory Fellow, Los Alamos National Laboratory
> Senior Scientist, New Mexico Consortium
> 100 Entrada Dr, Los Alamos, NM 87544
> Email: tterwilliger at newmexicoconsortium.org 
> <mailto:tterwilliger at newmexicoconsortium.org>
> Tel: 505-431-0010
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://phenix-online.org/pipermail/phenixbb/attachments/20210424/0a07737e/attachment.htm>