Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Additional fixes for downstream jobs #3187

Draft
wants to merge 20 commits into
base: develop
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from 16 commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
5974b96
Only archive gempak if it was run
DavidHuber-NOAA Dec 20, 2024
2649a86
Remove literal quotes
DavidHuber-NOAA Dec 20, 2024
a038cb9
Shellcheck stuff
DavidHuber-NOAA Dec 20, 2024
1180627
Remove duplicate marinebmat entry
DavidHuber-NOAA Dec 20, 2024
1efe711
Add DO_GEMPAK to archive environment
DavidHuber-NOAA Dec 20, 2024
4b7d39e
Merge remote-tracking branch 'emc/develop' into fix/awips
DavidHuber-NOAA Dec 20, 2024
4df2811
Fix fbwinds resource assignments
DavidHuber-NOAA Dec 24, 2024
939c238
Source preamble in gfs_bufr.sh
DavidHuber-NOAA Dec 24, 2024
66440fd
Merge remote-tracking branch 'emc/develop' into fix/awips
DavidHuber-NOAA Dec 30, 2024
52a1225
Remove duplicate preamble sourcing
DavidHuber-NOAA Dec 30, 2024
28e7775
Enable AWIPS for extended CI test
DavidHuber-NOAA Dec 30, 2024
0796a09
Source config.fbwnd if running AWIPS in forecast only mode
DavidHuber-NOAA Jan 2, 2025
6c3fb14
Merge branch 'develop' into fix/awips
aerorahul Jan 3, 2025
06bc9dd
Merge branch 'develop' into fix/awips
DavidHuber-NOAA Jan 6, 2025
66a0f86
Merge remote-tracking branch 'emc/develop' into fix/awips
DavidHuber-NOAA Jan 7, 2025
6d746b6
Merge remote-tracking branch 'emc/develop' into HEAD
DavidHuber-NOAA Jan 7, 2025
a1831e5
Apply multiple fixes for the gempakncdcupapgif job
DavidHuber-NOAA Jan 8, 2025
8187b7a
Address shellcheck issues
DavidHuber-NOAA Jan 8, 2025
1309cc0
Add missing braces
DavidHuber-NOAA Jan 8, 2025
1488020
Merge remote-tracking branch 'emc/develop' into fix/awips
DavidHuber-NOAA Jan 8, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion ci/cases/yamls/gfs_extended_ci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ base:
DO_GOES: "YES"
DO_BUFRSND: "YES"
DO_GEMPAK: "YES"
DO_AWIPS: "NO"
DO_AWIPS: "YES"
DO_NPOESS: "YES"
DO_GENESIS_FSU: "NO"
FCST_BREAKPOINTS: 192
Expand Down
5 changes: 0 additions & 5 deletions env/HERA.env
Original file line number Diff line number Diff line change
Expand Up @@ -133,11 +133,6 @@ elif [[ "${step}" = "marinebmat" ]]; then
export APRUNCFP="${launcher} -n \$ncmd --multi-prog"
export APRUN_MARINEBMAT="${APRUN_default}"

elif [[ "${step}" = "marinebmat" ]]; then

export APRUNCFP="${launcher} -n \$ncmd --multi-prog"
export APRUN_MARINEBMAT="${APRUN_default}"

elif [[ "${step}" = "marineanlvar" ]]; then

export APRUNCFP="${launcher} -n \$ncmd --multi-prog"
Expand Down
2 changes: 2 additions & 0 deletions parm/archive/gfs_downstream.yaml.j2
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,10 @@ gfs_downstream:
name: "GFS_DOWNSTREAM"
target: "{{ ATARDIR }}/{{ cycle_YMDH }}/gfs_downstream.tar"
required:
{% if DO_GEMPAK %}
- "{{ COMIN_ATMOS_GEMPAK | relpath(ROTDIR) }}/gfs_{{ cycle_YMDH }}.sfc"
- "{{ COMIN_ATMOS_GEMPAK | relpath(ROTDIR) }}/gfs_{{ cycle_YMDH }}.snd"
{% endif %}
{% for i in range(1, NUM_SND_COLLECTIVES) %}
- "{{ COMIN_ATMOS_BUFR | relpath(ROTDIR) }}/gfs_collective{{ i }}.fil"
{% endfor %}
Expand Down
1 change: 1 addition & 0 deletions parm/config/gfs/config.resources
Original file line number Diff line number Diff line change
Expand Up @@ -1288,6 +1288,7 @@ case ${step} in
ntasks=1
threads_per_task=1
memory="4GB"
tasks_per_node=1
;;

"mos_stn_prep")
Expand Down
2 changes: 1 addition & 1 deletion scripts/exgfs_atmos_postsnd.sh
Original file line number Diff line number Diff line change
Expand Up @@ -117,7 +117,7 @@ for fhr in "${hour_list[@]}"; do
echo "File ${filename} is required but not found."
err_exit "FATAL ERROR: logf${fhr} not found."
else
echo "${runscript} \"${fhr}\" \"${fhr_p}\" \"${FINT}\" \"${F00FLAG}\" \"${DATA}/${fhr}\"" >> "${DATA}/poescript_bufr"
echo "${runscript} ${fhr} ${fhr_p} ${FINT} ${F00FLAG} ${DATA}/${fhr}" >> "${DATA}/poescript_bufr"
fi
done

Expand Down
2 changes: 1 addition & 1 deletion scripts/exglobal_archive.py
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ def main():
'NMEM_ENS', 'DO_JEDIATMVAR', 'DO_VRFY_OCEANDA', 'FHMAX_FITS', 'waveGRD',
'IAUFHRS', 'DO_FIT2OBS', 'NET', 'FHOUT_HF_GFS', 'FHMAX_HF_GFS', 'REPLAY_ICS',
'OFFSET_START_HOUR', 'ARCH_EXPDIR', 'EXPDIR', 'ARCH_EXPDIR_FREQ', 'ARCH_HASHES',
'ARCH_DIFFS', 'SDATE', 'EDATE', 'HOMEgfs']
'ARCH_DIFFS', 'SDATE', 'EDATE', 'HOMEgfs', 'DO_GEMPAK']

archive_dict = AttrDict()
for key in keys:
Expand Down
15 changes: 8 additions & 7 deletions ush/gfs_bufr.sh
Original file line number Diff line number Diff line change
Expand Up @@ -21,16 +21,17 @@
# 2024-08-08 Bo Cui: Update to handle one forecast at a time
# echo "History: February 2003 - First implementation of this utility script"
#
fhr="$1"
fhr_p="$2"
FINT="$3"
F00FLAG="$4"
workdir="$5"

cd "${workdir}" || exit 2

source "${USHgfs}/preamble.sh"

fhr="${1}"
fhr_p="${2}"
FINT="${3}"
F00FLAG="${4}"
workdir="${5}"

cd "${workdir}" || exit 2

if [[ "${F00FLAG}" == "YES" ]]; then
f00flag=".true."
else
Expand Down
2 changes: 1 addition & 1 deletion workflow/applications/gfs_cycled.py
Original file line number Diff line number Diff line change
Expand Up @@ -119,7 +119,7 @@ def _get_app_configs(self, run):
configs += ['postsnd']

if options['do_awips']:
configs += ['awips']
configs += ['awips', 'fbwind']
aerorahul marked this conversation as resolved.
Show resolved Hide resolved

if options['do_wave']:
configs += ['waveinit', 'waveprep', 'wavepostsbs', 'wavepostpnt']
Expand Down
2 changes: 1 addition & 1 deletion workflow/applications/gfs_forecast_only.py
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ def _get_app_configs(self, run):
configs += ['gempak']

if options['do_awips']:
configs += ['awips']
configs += ['awips', 'fbwind']

if options['do_ocean'] or options['do_ice']:
configs += ['oceanice_products']
Expand Down
1 change: 1 addition & 0 deletions workflow/hosts/hera.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ LOCALARCH: 'NO'
ATARDIR: '/NCEPDEV/${HPSS_PROJECT}/1year/${USER}/${machine}/scratch/${PSLOT}'
MAKE_NSSTBUFR: 'NO'
MAKE_ACFTBUFR: 'NO'
DO_AWIPS: 'NO'
SUPPORTED_RESOLUTIONS: ['C1152', 'C768', 'C384', 'C192', 'C96', 'C48']
COMINecmwf: /scratch1/NCEPDEV/global/glopara/data/external_gempak/ecmwf
COMINnam: /scratch1/NCEPDEV/global/glopara/data/external_gempak/nam
Expand Down
1 change: 1 addition & 0 deletions workflow/hosts/hercules.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ MAKE_NSSTBUFR: 'NO'
MAKE_ACFTBUFR: 'NO'
DO_TRACKER: 'NO'
DO_GENESIS: 'NO'
DO_AWIPS: 'NO'
SUPPORTED_RESOLUTIONS: ['C1152', 'C768', 'C384', 'C192', 'C96', 'C48']
COMINecmwf: /work/noaa/global/glopara/data/external_gempak/ecmwf
COMINnam: /work/noaa/global/glopara/data/external_gempak/nam
Expand Down
1 change: 1 addition & 0 deletions workflow/hosts/orion.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ MAKE_NSSTBUFR: 'NO'
MAKE_ACFTBUFR: 'NO'
DO_TRACKER: 'NO'
DO_GENESIS: 'NO'
DO_AWIPS: 'NO'
SUPPORTED_RESOLUTIONS: ['C1152', 'C768', 'C384', 'C192', 'C96', 'C48']
COMINecmwf: /work/noaa/global/glopara/data/external_gempak/ecmwf
COMINnam: /work/noaa/global/glopara/data/external_gempak/nam
Expand Down
2 changes: 1 addition & 1 deletion workflow/rocoto/gfs_tasks.py
Original file line number Diff line number Diff line change
Expand Up @@ -1410,7 +1410,7 @@ def fbwind(self):
deps.append(rocoto.add_dependency(dep_dict))
dependencies = rocoto.create_dependency(dep=deps, dep_condition='and')

resources = self.get_resource('awips')
resources = self.get_resource('fbwind')
# TODO: It would be better to use task dependencies on the
# individual post jobs rather than data dependencies to avoid
# prematurely starting with partial files. Unfortunately, the
Expand Down