[phenixbb] temp files

[Billy Poon]

Hi James,

By default, the GUI should try to delete AutoBuild temporary files.
However, if the jobs crash, the cleaning up might not take place. In that
case, the deletion of temporary files can be manually run by selecting
"Projects" in the menu bar, then "Clean up large files", and then selecting
the project directory.

--
Billy K. Poon
Research Scientist, Molecular Biophysics and Integrated Bioimaging
Lawrence Berkeley National Laboratory
1 Cyclotron Road, M/S 33R0345
Berkeley, CA 94720
Fax: (510) 486-5909
Web: https://phenix-online.org


On Sat, Apr 24, 2021 at 2:56 PM James Holton <jmholton at lbl.gov> wrote:

> What if people are using the GUI ?
>
> On 4/24/2021 2:51 PM, Tom Terwilliger wrote:
>
> Hi James,
>
> Not the code (I hope), just any scripts that use these methods. Like this:
>
> mkdir /var/tmp/autosol
> phenix.autosol p9.sca 2 se temp_dir=/var/tmp/autosol
>
> Now the temp files go in /var/tmp/autosol and the output files go in
> AutoSol_run_xxx/ as usual
>
> All the best,
> Tom
>
> All the best,
> Tom T
>
> On Sat, Apr 24, 2021 at 11:59 AM James Holton <jmholton at lbl.gov> wrote:
>
>> Thank you Tom!
>>
>> Ok.  So, in order to change the default I need to go through the code
>> looking for "temp_dir" and change things?
>>
>>
>> On 4/24/2021 10:16 AM, Tom Terwilliger wrote:
>>
>> Hi James,
>>
>> There is no overall Phenix temp directory specification, but most of the
>> temp_dir usage is from autosol/autobuild/ligandfit/map_to_model.  Each of
>> these has the keyword "temp_dir=xxxx" which you should be
>> able to set to any directory you want (and local is better as you note).
>> Most programs using a temp_dir also have a keyword clean_up=True as well.
>>
>> All the best,
>> Tom T
>>
>> On Sat, Apr 24, 2021 at 10:51 AM James Holton <jmholton at lbl.gov> wrote:
>>
>>> Thank you Li-Wei
>>>
>>> Definitely not placing blame on one program. Phenix.autobuild is another
>>> big temp file producer.  So is XDS. Clearly this ligand run was a case
>>> of a misconfigured, runaway task that never finished. However, the files
>>> lingered on disk, eating up inodes for 3 years!
>>>
>>> The reason I'm asking is I think there are significant performance
>>> increases to be gained by using fast, local storage for scratch files.
>>> This is not just in speed but storage and overall system/cluster
>>> performance.  Very few things are more expensive than an NFS write!
>>>
>>> Does anyone know how to change the default temp file location across
>>> phenix ?  Is this a cctbx thing?
>>>
>>> Thanks
>>>
>>> -James
>>>
>>>
>>> On 4/23/2021 9:38 PM, Li-Wei Hung wrote:
>>> > Hi James,
>>> >
>>> > I'll leave the global Phenix temp aspect to Billy.
>>> > For ligand identification specifically, the working directory is where
>>> > all the files are located.  The program will purge most of the
>>> > intermediate files upon completion. If the user interrupted the runs
>>> > or if the program crashed at certain spots, the purge mechanism might
>>> > not kick in. Even so, it'd take many runs to accumulate 20e6 (2e7?)
>>> > files. In any case, you've got a point and I'll look into salvaging
>>> > intermediate files of ligand identification as soon as they are not
>>> > needed in the process.
>>> >
>>> > Thanks,
>>> >
>>> > Li-Wei
>>> >
>>> > On 4/23/2021 7:03 PM, James Holton wrote:
>>> >> Hello all,
>>> >>
>>> >> Is there a way to configure phenix at install time (or perhaps
>>> >> post-install) to put temporary files under /tmp ?  I just had to
>>> >> delete 20e6 temp files over NFS from a single user's phenix ligand
>>> >> identification run.  The delete took almost a month.
>>> >>
>>> >> Apologies if I am neglecting to look somewhere obvious in the
>>> >> documentation,
>>> >>
>>> >> Happy Weekend!
>>> >>
>>> >> -James Holton
>>> >> MAD Scientist
>>> >>
>>> >> _______________________________________________
>>> >> phenixbb mailing list
>>> >> phenixbb at phenix-online.org
>>> >> http://phenix-online.org/mailman/listinfo/phenixbb
>>> >> Unsubscribe: phenixbb-leave at phenix-online.org
>>> >
>>>
>>> _______________________________________________
>>> phenixbb mailing list
>>> phenixbb at phenix-online.org
>>> http://phenix-online.org/mailman/listinfo/phenixbb
>>> Unsubscribe: phenixbb-leave at phenix-online.org
>>
>>
>>
>> --
>> Thomas C Terwilliger
>> Laboratory Fellow, Los Alamos National Laboratory
>> Senior Scientist, New Mexico Consortium
>> 100 Entrada Dr, Los Alamos, NM 87544
>> Email: tterwilliger at newmexicoconsortium.org
>> Tel: 505-431-0010
>>
>>
>>
>
> --
> Thomas C Terwilliger
> Laboratory Fellow, Los Alamos National Laboratory
> Senior Scientist, New Mexico Consortium
> 100 Entrada Dr, Los Alamos, NM 87544
> Email: tterwilliger at newmexicoconsortium.org
> Tel: 505-431-0010
>
>
> _______________________________________________
> phenixbb mailing list
> phenixbb at phenix-online.org
> http://phenix-online.org/mailman/listinfo/phenixbb
> Unsubscribe: phenixbb-leave at phenix-online.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://phenix-online.org/pipermail/phenixbb/attachments/20210426/283989ad/attachment.htm>