<div dir="ltr">Hi James,<div><br></div><div>By default, the GUI should try to delete AutoBuild temporary files. However, if the jobs crash, the cleaning up might not take place. In that case, the deletion of temporary files can be manually run by selecting "Projects" in the menu bar, then "Clean up large files", and then selecting the project directory.</div><div><br clear="all"><div><div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div><div>--</div><div><span style="font-size:12.8000001907349px">Billy K. Poon</span><br></div></div><div>Research Scientist, Molecular Biophysics and Integrated Bioimaging</div><div>Lawrence Berkeley National Laboratory</div><div>1 Cyclotron Road, M/S 33R0345</div><div>Berkeley, CA 94720</div><div>Fax: (510) 486-5909</div><div>Web: <a href="https://phenix-online.org" target="_blank">https://phenix-online.org</a></div></div></div></div></div></div></div></div></div></div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sat, Apr 24, 2021 at 2:56 PM James Holton <<a href="mailto:jmholton@lbl.gov">jmholton@lbl.gov</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex">
<div>
What if people are using the GUI ?<br>
<br>
<div>On 4/24/2021 2:51 PM, Tom Terwilliger
wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Hi James,
<div><br>
<div>Not the code (I hope), just any scripts that use these
methods. Like this:</div>
<div><br>
</div>
<div>mkdir /var/tmp/autosol</div>
<div>phenix.autosol p9.sca 2 se temp_dir=/var/tmp/autosol</div>
<div><br>
</div>
<div>Now the temp files go in /var/tmp/autosol and the output
files go in AutoSol_run_xxx/ as usual</div>
<div><br>
</div>
<div>All the best,</div>
<div>Tom</div>
<div><br>
</div>
<div>All the best,</div>
<div>Tom T</div>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Sat, Apr 24, 2021 at 11:59
AM James Holton <<a href="mailto:jmholton@lbl.gov" target="_blank">jmholton@lbl.gov</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex">
<div> Thank you Tom!<br>
<br>
Ok. So, in order to change the default I need to go through
the code looking for "temp_dir" and change things?<br>
<br>
<br>
<div>On 4/24/2021 10:16 AM, Tom Terwilliger wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Hi James,
<div><br>
</div>
<div>There is no overall Phenix temp directory
specification, but most of the temp_dir usage is from
autosol/autobuild/ligandfit/map_to_model. Each of
these has the keyword "temp_dir=xxxx" which you should
be</div>
<div>able to set to any directory you want (and local is
better as you note). Most programs using a temp_dir
also have a keyword clean_up=True as well.</div>
<div><br>
</div>
<div>All the best,</div>
<div>Tom T</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Sat, Apr 24, 2021
at 10:51 AM James Holton <<a href="mailto:jmholton@lbl.gov" target="_blank">jmholton@lbl.gov</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex">Thank you Li-Wei<br>
<br>
Definitely not placing blame on one program.
Phenix.autobuild is another <br>
big temp file producer. So is XDS. Clearly this
ligand run was a case <br>
of a misconfigured, runaway task that never finished.
However, the files <br>
lingered on disk, eating up inodes for 3 years!<br>
<br>
The reason I'm asking is I think there are significant
performance <br>
increases to be gained by using fast, local storage
for scratch files. <br>
This is not just in speed but storage and overall
system/cluster <br>
performance. Very few things are more expensive than
an NFS write!<br>
<br>
Does anyone know how to change the default temp file
location across <br>
phenix ? Is this a cctbx thing?<br>
<br>
Thanks<br>
<br>
-James<br>
<br>
<br>
On 4/23/2021 9:38 PM, Li-Wei Hung wrote:<br>
> Hi James,<br>
><br>
> I'll leave the global Phenix temp aspect to
Billy.<br>
> For ligand identification specifically, the
working directory is where <br>
> all the files are located. The program will
purge most of the <br>
> intermediate files upon completion. If the user
interrupted the runs <br>
> or if the program crashed at certain spots, the
purge mechanism might <br>
> not kick in. Even so, it'd take many runs to
accumulate 20e6 (2e7?) <br>
> files. In any case, you've got a point and I'll
look into salvaging <br>
> intermediate files of ligand identification as
soon as they are not <br>
> needed in the process.<br>
><br>
> Thanks,<br>
><br>
> Li-Wei<br>
><br>
> On 4/23/2021 7:03 PM, James Holton wrote:<br>
>> Hello all,<br>
>><br>
>> Is there a way to configure phenix at install
time (or perhaps <br>
>> post-install) to put temporary files under
/tmp ? I just had to <br>
>> delete 20e6 temp files over NFS from a single
user's phenix ligand <br>
>> identification run. The delete took almost a
month.<br>
>><br>
>> Apologies if I am neglecting to look
somewhere obvious in the <br>
>> documentation,<br>
>><br>
>> Happy Weekend!<br>
>><br>
>> -James Holton<br>
>> MAD Scientist<br>
>><br>
>>
_______________________________________________<br>
>> phenixbb mailing list<br>
>> <a href="mailto:phenixbb@phenix-online.org" target="_blank">phenixbb@phenix-online.org</a><br>
>> <a href="http://phenix-online.org/mailman/listinfo/phenixbb" rel="noreferrer" target="_blank">http://phenix-online.org/mailman/listinfo/phenixbb</a><br>
>> Unsubscribe: <a href="mailto:phenixbb-leave@phenix-online.org" target="_blank">phenixbb-leave@phenix-online.org</a><br>
><br>
<br>
_______________________________________________<br>
phenixbb mailing list<br>
<a href="mailto:phenixbb@phenix-online.org" target="_blank">phenixbb@phenix-online.org</a><br>
<a href="http://phenix-online.org/mailman/listinfo/phenixbb" rel="noreferrer" target="_blank">http://phenix-online.org/mailman/listinfo/phenixbb</a><br>
Unsubscribe: <a href="mailto:phenixbb-leave@phenix-online.org" target="_blank">phenixbb-leave@phenix-online.org</a></blockquote>
</div>
<br clear="all">
<div><br>
</div>
-- <br>
<div dir="ltr">
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div dir="ltr">Thomas C Terwilliger
<div>Laboratory Fellow, Los Alamos
National Laboratory</div>
<div>Senior Scientist, New Mexico
Consortium</div>
<div>100 Entrada Dr, Los Alamos, NM
87544</div>
<div>Email: <a href="mailto:tterwilliger@newmexicoconsortium.org" target="_blank">tterwilliger@newmexicoconsortium.org</a></div>
<div>Tel: 505-431-0010</div>
<div><br>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
<br>
</div>
</blockquote>
</div>
<br clear="all">
<div><br>
</div>
-- <br>
<div dir="ltr">
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div dir="ltr">Thomas C Terwilliger
<div>Laboratory Fellow, Los Alamos National
Laboratory</div>
<div>Senior Scientist, New Mexico Consortium</div>
<div>100 Entrada Dr, Los Alamos, NM 87544</div>
<div>Email: <a href="mailto:tterwilliger@newmexicoconsortium.org" target="_blank">tterwilliger@newmexicoconsortium.org</a></div>
<div>Tel: 505-431-0010</div>
<div><br>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
<br>
</div>
_______________________________________________<br>
phenixbb mailing list<br>
<a href="mailto:phenixbb@phenix-online.org" target="_blank">phenixbb@phenix-online.org</a><br>
<a href="http://phenix-online.org/mailman/listinfo/phenixbb" rel="noreferrer" target="_blank">http://phenix-online.org/mailman/listinfo/phenixbb</a><br>
Unsubscribe: <a href="mailto:phenixbb-leave@phenix-online.org" target="_blank">phenixbb-leave@phenix-online.org</a></blockquote></div>