Questions about wind plant cost data in raw FERC1

[aesharpe]

Reposting some questions received via email in case anyone else might find the answers useful!

---

1. How do I find a dataset like f1_gnrt_plant for large plants? I am very interested in finding the plant cost of all wind projects post-2000
2. Is the best method to link to the f1 data the individual plant_id_eia?
3. What do you recommend for linking an existing set of substations and lmp nodes to the f1_substations dataset and the lmp price in PUDL?
4. What is the best method for matching plant_id_eia to facility_id in the hourly_emissions_epacems dataset?

---

1. How do I find a dataset like f1_gnrt_plant for large plants? I am very interested in finding the plant cost of all wind projects post-2000

f1_steam is the large plants equivalent. It may have steam in the name, but there are all sorts of plant types in there (I wish I knew why). The other plant tables are f1_hydro and f1_pumped_storage, but I doubt you'll use those for wind!

2. Is the best method to link to the f1 data the individual plant_id_eia?

I would strongly recommend against using the raw FERC1 data for this. The PUDL db has all of the FERC1 plant tables integrated and cleaned so that there are plant IDs and fewer disparate plant types. See the plants_steam_ferc1 (f1_steam) and plants_small_ferc1 (f1_gnrt_plant). The dev branch even has a plants_all_ferc1 output table that combines all the plant tables from FERC1. All the FERC1 plant tables in the PUDL db have a plant_id_pudl field that you can group by along with year to get plant values by year. Note that plant_id_ferc1, also included in the FERC1 plant tables in the PUDL db, is technically a sub-plant label (because FERC doesn't report at any pre-specified level). plant_id_pudl links directly to plant_id_eia.

FERC is weird, so here are some important things to be aware of (whether using the raw or PUDL db data):

• There may be total rows embedded in the data!! (yikes, double counting. We're hoping to fix this soon!)
• The plant_kind (raw FERC1 db) or plant_type, fuel_kind (PUDL db) columns are incomplete and somewhat janky (largely because FERC has zero standardization). We are in the process of hand-mapping all FERC and EIA records on a record by record basis, not just plant by plant. EIA has very thorough technology description fields, so ultimately we'll be able to map FERC records to those cleaner technology types. But that's still very much in the works.
• Sub-plant components may exist in multiple tables. If you group by plant_id_pudl in the steam table you may be missing some smaller components of the same plant in the small plants table (or vice versa). This is why the plants_all_ferc1 table is useful.
• Only large utilities are reporting to FERC. Check out the qualifications on our FERC1 documentation page. This might mean that smaller independent wind producers are not included in the data.

3. What do you recommend for linking an existing set of substations and lmp nodes to the f1_substations dataset and the lmp price in PUDL?

I haven't worked with the f1_substations data yet, so I'm not sure!

4. What is the best method for matching plant_id_eia to facility_id in the hourly_emissions_epacems dataset?

This one is good timing. I'm in the process of integrating the EPA's new crosswalk to EIA data (see #1692 ) into PUDL. Our published parquet of the EPA CEMS data has the following columns (among others):

• plant_id_eia: This is close but not quite to what you'll find in EIA. The EPA has a few discrepancies that are fixed by the crosswalk. If you merge the CEMS data with EIA data on the field of the same name right now it would be very close but not perfect (we fix this in the new version coming soon!).
• unitid: This is the EPA's emissions unit (usually correlates with boilers or gas turbines, i.e., where the emissions are created)
• facility_id: This is the EPA's internal business database id, we are removing it from the published output because it doesn't provide any new information or helpful links to other data.
• unit_id: This is a unique identifier fro unitid. Units often have repeating names like 1, 2, 3, so this field assigns each of them a unique value. This field is also primarily used for internal processing at EPA and will be removed from future publications.
  I would recommend waiting for the crosswalk to get published on our end (very soon if you're working with our dev branch!) Otherwise, you can download it yourself.

This is all to say that plant_id_eia is already in the CEMS data (you can ignore facility_id) and if you want the most accurate version of plant_id_eia wait until the crosswalk has been integrated!

[zaneselvans]

If you want to see the cleaned up PUDL versions of these FERC1 plant tables, you can browse them on our Datasette deployment:

• plants_steam_ferc1
• plants_small_ferc1

Unfortunately, there's very little information about wind and other renewable power generation in the FERC data, probably in large part because those types of plants tend to be owned and operated by IPPs (independent power producers), which have PPAs (power purchase agreements) with the utilities.

We don't have LMP prices in PUDL, but there are a couple of related columns in the generators_entity_eia table, namely rto_iso_lmp_node_id and rto_iso_lmp_wholesale_reporting_id that might be helpful.

There's also a bunch of renewable energy PPA data buried in the massive FERC EQR dataset, which we're in the process of integrating, but it will take some processing to really make sense of.

[TrentonBush]

Copy/paste my email response:

Your questions 1 and 2 involve raw FERC Form 1 data. Though Catalyst makes this dataset public, the raw data is in rough shape and takes significant effort to clean and standardize to the level most people expect from a relational database. I think of the raw FERC data more as a digitized stack of paper forms than as a true database. For this reason, we have integrated only select subsets of this information into our higher quality PUDL database. Luckily this includes the f1_steam and f1_gnrt_plant tables that contain the plant financial information you're looking for. See our FERC Form 1 overview for more information.

Here are more specific answers to your questions:

1. Despite the name, the f1_steam table contains info about large plants of many types, including a few hundred wind and solar facilities. As far as covering "all wind projects", unfortunately much of that financial info may not exist publicly. To put numbers to it, I count 1217 wind plants with 118 GW capacity in 2020 EIA data, but only 678 plants with 68 GW capacity in FERC data. FERC only regulates wholesale electricity transactions that cross state lines and is only entitled to collect information relevant to those activities. This is in contrast to the EIA, whose mandate is to collect a bunch of energy data (not financial!) for informational purposes. Utilities that don't operate between states may not fall under FERC's jurisdiction. Examples include smaller vertically integrated utilities in the West and Southeast or, of particular relevance to you, much of ERCOT. Additionally, plant owners can apply for and receive Exempt Wholesale Generator status, which exempts them from a lot of financial reporting.
   1. The best built-in way to link f1 plants to EIA plants is via the plant_id_pudl field. The PUDL table plants_ferc1 maps f1 plants to pudl IDs, and then the table plants_eia maps pudl IDs to EIA IDs.
2. We have not integrated substation data into PUDL, so I don't have direct experience here. I don't know what your existing dataset looks like, but in general you are probably looking at a "record linkage" problem, where you need to construct a kind of similarity score between rows based on their overlapping information, then pick the best matches. This will not be 100% accurate. We use these techniques, supplemented with manual corrections, when linking plants and utilities between FERC and EIA. PUDL does not have LMP prices.
3. We are actively working on better EPA CEMS integration that should be released in the next couple of weeks. It is based on an EPA-EIA mapping created by the EPA. You can access that mapping at the EPA's GitHub page.