Recovering prev_outdir from within a flow

[hironori-kondo]

I'm looking at @tomdemeyere's grid_phonon_flow() implementation. I'm hoping to run a job in parallel with the ph.x that uses the relaxation "dir_name" as its prev_outdir. From what I can see, there's no easy way to reference this "dir_name" from outside the flow, but I am no programmer---could someone kindly confirm whether this is the case? If so, I may have to write my own flow.

[Andrew-S-Rosen]

Hi @nerdwarrior. Thanks for your question. Could you clarify your question a bit more? If I understand correctly, you want to run quacc.recipes.espresso.phonons.grid_phonon_flow but you want the prev_outdir of one or more of the jobs (which job?) in the flow to be the directory of relax_job.

[hironori-kondo]

Thank you for the prompt reply! I want to grab the directory of the quacc.recipes.expresso.core.relax_job called in quacc.recipes.espresso.phonons.grid_phonon_flow and use that as the prev_outdir for a quacc.recipes.expresso.core.non_scf_job. The directory is used internally within the flow as the prev_outdir of the phonon calculations, but I would like to run a non scf in parallel with the phonon.

[tomdemeyere]

Just clarifying a few things here:

First thing: prev_outdir can be specified to some specific jobs, and as a result, the QE calculations will run using this directory. This is in contrast to copy_files which can be used to copy the files to the folder where the QE will run (a new one in this case)

The former approach has the advantage that it does not need to copy files around, this is useful for a small job that needs huge amounts of data (pp.x ...). Unfortunately, the drawback is that this approach is often not compatible with running things concurrently. This is due to the fact that multiple calculations will then write at the same time in the same prev_outdir. You can see the list of acceptable situations here.

In the case of the grid_phonon_flow() the copy_files approach is used.

Second thing: Are you sure that grid_phonon_flow is what you want? This approach is used to parallelize the phonon calculation with respect to q-points and representations. If yes, then the directory returned by this flow (in the results dict) should contain everything needed to perform an additional calculation.

[tomdemeyere]

I'm running O(100) atom unit cells in an HPC environment. Please let me know if I am misguided.

If you are not using a workflow engine, you are misguided, this approach will have little advantage. If you are, then this method will make the overall calculation much faster.

As for prev_outdir vs copy_files, I think I understand---the prev_outdir you're passing to the ph_init_job is only used for copying later, correct? Similarly, if I want access to the relaxation outputs for a concurrent non_scf_job, I should be using copy_files so as to prevent conflicts.

In the ph_init_job prev_outdir is used because no competition is possible here, only this job is running. Actually you can see that after prev_outdir is retrieved and used as copy_files in the subsequent calculations in the for loop (in _grid_phonon_subflow), at which point we switch to copy_files because jobs might run concurrently.

Something important from this: the files generated during the relax_job are copied over until the very last step (in fact they are needed for the ph.x calculations), hence you can use the final folder which should contain the necessary files (the charge density) for your non-scf calculation.

Further, in the copy_files case, wouldn't I still need access to the relax_job output directory if I wanted to run a job current to the grid-parallelized phonon calculations? It seems that using the results dict returned by the flow would require further calculations to follow, not run concurrently with, the phonon calculations.

If you want to run your non-scf calculation concurrently with the rest of the ph.x workflow well... you will have to write your own flow yes. Although (@Andrew-S-Rosen) I would argue that the mandatory relax in grid_phonon_flow is quite invasive, without it you can just write a flow like:

def my_flow(...):
    relaxed_atoms_results = relax_job...
    grid_phonon_results = grid_phonon(relaxed_atoms_results["atoms"])
    non_scf_results = non_scf_job(relaxed_atoms_results["atoms"], copy_files=relaxed_atoms_results[...])

In this case the non_scf_job does not depend on grid_phonon_results and will run concurrently

PS: In theory you could "hack" the flow and provide atoms already optimised, then pass calculation = nscf to the relax job, not the intended usage...

[Andrew-S-Rosen]

Thank you for chiming in, @tomdemeyere! Everything you said is correct. @nerdwarrior: let me know if it is still unclear. If you aren't using a workflow engine, you can simply do relax_job and then a phonon_job without any concern.

@tomdemeyere: I agree that requiring a relax job in grid_phonon_flow is undesirable. At some point, that should be made optional.

[hironori-kondo]

If you are not using a workflow engine, you are misguided, this approach will have little advantage. If you are, then this method will make the overall calculation much faster.

I am using a workflow engine, so that's fantastic!

In the ph_init_job prev_outdir is used because no competition is possible here, only this job is running. Actually you can see that after prev_outdir is retrieved and used as copy_files in the subsequent calculations in the for loop (in _grid_phonon_subflow), at which point we switch to copy_files because jobs might run concurrently.

Something important from this: the files generated during the relax_job are copied over until the very last step (in fact they are needed for the ph.x calculations), hence you can use the final folder which should contain the necessary files (the charge density) for your non-scf calculation.

I follow, thank you for the clarification!

If you want to run your non-scf calculation concurrently with the rest of the ph.x workflow well... you will have to write your own flow yes. Although (@Andrew-S-Rosen) I would argue that the mandatory relax in grid_phonon_flow is quite invasive, without it you can just write a flow like:

def my_flow(...):
    relaxed_atoms_results = relax_job...
    grid_phonon_results = grid_phonon(relaxed_atoms_results["atoms"])
    non_scf_results = non_scf_job(relaxed_atoms_results["atoms"], copy_files=relaxed_atoms_results[...])

In this case the non_scf_job does not depend on grid_phonon_results and will run concurrently

This makes sense, and I am inclined to agree re: invasiveness, as the phonon execution seems quite distinct.

Thank you very much, @tomdemeyere ---this was super helpful. @Andrew-S-Rosen, I am all set. I appreciate everyone taking time out of their weekend.

[Andrew-S-Rosen]

@nerdwarrior: Of course!

Regarding the relaxation step, I'll just note that the patch should not be too painful. It would probably amount to adding a run_relax: bool = True keyword argument to grid_phonon_flow and then modifying the function to only run the relaxation step if run_relax is True:

    if run_relax:
        pw_job_results = pw_job(atoms)
        prev_outdir = pw_job_results["dir_name"]

    ph_init_job_results = ph_init_job(prev_outdir=prev_outdir)  # I guess you'd also need to include a `prev_outdir` keyword argument in `grid_phonon_flow` so there is a value to be used here if `run_relax` is `False`

Feel free to submit a PR sometime if you wish.

[hironori-kondo]

@Andrew-S-Rosen I will try to take a crack at it later today!