diff --git a/data/clean/latest-created.rss b/data/clean/latest-created.rss index c9525f6..21a7ae0 100644 --- a/data/clean/latest-created.rss +++ b/data/clean/latest-created.rss @@ -1,2 +1,2 @@ -New datasets from opendata.cityofnewyork.ushttps://github.com/palewire/nyc-open-data-monitorThe latest datasets posted to New York City's data portalhttp://www.rssboard.org/rss-specificationpython-feedgenTue, 28 Nov 2023 00:16:59 +0000Local Law 83 - City Agency Advertising Spendhttps://data.cityofnewyork.us/d/n56u-3hzvPer Local Law 83 of 2021, the Mayor's Office of Ethnic and Community Media is required to report annually on each agency's full advertising spend across all media categories, including ethnic and community (ECM), mainstream, out-of-home, social media, etc. This dataset reflects the raw data that MOECM received from City Agencies on their annual advertising spend. For more information, please visit the <a href="https://www.nyc.gov/site/ethnicmedia/index.page">MOECM website</a>.n56u-3hzvCity GovernmentFri, 24 Nov 2023 17:21:13 +0000Parking Violations Issued - Fiscal Year 2023https://data.cityofnewyork.us/d/869v-vr48This dataset provides data on Parking Violations Issued between July 1, 2022 to June 30, 2023. In New York City, the fiscal year begins on July 1st of one calendar year and ends on June 30th of the following calendar year. Click here to find out more about the NYC Fiscal Year. Parking Violations Issuance datasets contain violations issued during the respective fiscal year. The Issuance datasets are not updated to reflect violation status, the information only represents the violation(s) at the time they are issued. Since appearing on an issuance dataset, a violation may have been paid, dismissed via a hearing, statutorily expired, or had other changes to its status. To see the current status of outstanding parking violations, please look at the Open Parking & Camera Violations dataset. <b>NOTE:</b> This dataset includes the records from the first month of FY2024. You may find all of the Parking Violations Issued since fiscal year 2014 in this dataset collection.869v-vr48City GovernmentTue, 14 Nov 2023 14:55:34 +0000Taxi and For Hire Vehicle (FHV) Relief Stands (Map)https://data.cityofnewyork.us/d/udk9-t2yfThis data details the location of the Taxi and For Hire Vehicle(FHV) stands and the number of spaces provided. Taxi and FHV relief stands allow drivers to park their vehicles for up to one hour. This gives drivers the opportunity to leave their vehicles and take care of personal needs. <b>TNOTE:</b> Taxi and FHV relief stands should not be confused with taxi stands. Taxi stands allow drivers to wait in their cars to pick up passengers.udk9-t2yfTransportationFri, 27 Oct 2023 19:11:50 +0000DOB Safety Violationshttps://data.cityofnewyork.us/d/855j-jadyThis data set includes violations issued on devices through the New York City Department of Buildings' DOB NOW: Safety Violations module. The data is collected because the Department of Buildings tracks violation issuance and related information. This data include items such as violations number, violations type, violation issuance data, and BIN.855j-jadyHousing & DevelopmentWed, 18 Oct 2023 16:22:08 +0000Open Streets Locations Maphttps://data.cityofnewyork.us/d/6vys-sfk5New York City's Open Streets program transforms streets into public spaces open to all. These public spaces allows a variety of activities including education and cultural programming as well as building communities. For more information please visit: https://www.nyc.gov/html/dot/html/pedestrians/openstreets.shtml Please note: there may be a delay in updates to the Open Data files.6vys-sfk5HealthTue, 17 Oct 2023 19:18:56 +0000Mayor's Management Report Spending and Budgethttps://data.cityofnewyork.us/d/2jrp-puwzContains Spending and Budget information for City agencies, as reported in the Mayoral Management Report (MMR).2jrp-puwzCity GovernmentMon, 25 Sep 2023 14:06:18 +0000Mayor's Management Report Agency Resourceshttps://data.cityofnewyork.us/d/4qmi-txnkIncludes NYC agency resources from the Mayor's Management Report (MMR), such as expenditures (includes all funds), personnel, revenue and paid overtime. This data is an overview of the financial and workforce resources used by an agency and the planned resources available to the agency in a given reporting fiscal year or future fiscal years. The MMR, archived reports, and additional information is available at: https://www1.nyc.gov/site/operations/performance/mmr.page and https://dmmr.nyc.gov/4qmi-txnkCity GovernmentFri, 22 Sep 2023 17:03:49 +0000Building Elevation and Subgrade (BES) - Maphttps://data.cityofnewyork.us/d/nmzg-6q5gThe Building Elevation and Subgrade data contains New York City building centroids derived from the Department of Building's (DOB) February 26th, 2022 building footprint dataset. Each record contains a grade and first floor measurement for each building (recorded as feet above sea-level in the NADV88 vertical datum) and indicates if subgrade space exists. DCP contracted with an external data vendor to generate a single point, or centroid, that represented the location of the center of every building recorded in the DOB dataset. The dataset excluded the footprints of small accessory buildings such as sheds. Each row within the dataset represents one building centroid, and records the X and Y coordinates of that centroid in the NAD 1983 coordinate system.nmzg-6q5gCity GovernmentTue, 19 Sep 2023 20:47:20 +00002023 Open Data Plan: Civic Engagement Activitieshttps://data.cityofnewyork.us/d/xw44-6wmwReleased as a part of the 2023 Open Data Plan, this is a list of civic engagement activities reported by city agencies for the purpose of providing context, spreading awareness and increasing the use of public city data. Details include activity descriptions, date of completion and resource links to further information on the reported activity.xw44-6wmwCity GovernmentTue, 12 Sep 2023 21:27:08 +0000Building Elevation and Subgrade (BES)https://data.cityofnewyork.us/d/bsin-59hvThe Building Elevation and Subgrade data contains New York City building centroids derived from the Department of Building's (DOB) February 26th, 2022 building footprint dataset. Each record contains a grade and first floor measurement for each building (recorded as feet above sea-level in the NADV88 vertical datum) and indicates if subgrade space exists. DCP contracted with an external data vendor to generate a single point, or centroid, that represented the location of the center of every building recorded in the DOB dataset. The dataset excluded the footprints of small accessory buildings such as sheds. Each row within the dataset represents one building centroid, and records the X and Y coordinates of that centroid in the NAD 1983 coordinate system.bsin-59hvCity GovernmentTue, 12 Sep 2023 19:06:08 +00002023 Open Data Plan: Removalshttps://data.cityofnewyork.us/d/bhst-9d3fThe list of the datasets that were removed from the Open Data Plan.bhst-9d3fCity GovernmentMon, 11 Sep 2023 23:33:38 +00002023 Open Data Plan: FOIL Datasetshttps://data.cityofnewyork.us/d/93kn-fhqtThe collection of datasets identified via FOIL reporting and their respective release statuses as a part of the 2023 Open Data Plan.93kn-fhqtCity GovernmentMon, 11 Sep 2023 17:42:23 +00002023 Open Data Plan: Future Releaseshttps://data.cityofnewyork.us/d/59qh-uc8yThe collection of datasets set to be released on the Open Data Portal, according to the 2023 Open Data Plan.59qh-uc8yCity GovernmentMon, 11 Sep 2023 17:26:03 +00002023 Open Data Plan: FOIL Metricshttps://data.cityofnewyork.us/d/33mm-wzxhVarious FOIL reporting metrics as part of the 2023 Open Data Compliance Plan.33mm-wzxhCity GovernmentMon, 11 Sep 2023 16:55:54 +00002023 Open Data Plan: Website Datahttps://data.cityofnewyork.us/d/rzp5-dkkiData on public websites maintained by or on behalf of the city agencies.rzp5-dkkiCity GovernmentMon, 11 Sep 2023 15:44:31 +0000Preliminary Mayor's Management Report Agency Resourceshttps://data.cityofnewyork.us/d/nvzu-6t9yIncludes NYC agency resources from the Preliminary Mayor's Management Report (PMMR), such as expenditures (includes all funds), personnel, revenue and paid overtime. This data is an overview of the financial and workforce resources used by an agency and the planned resources available to the agency in a given reporting fiscal year or future fiscal years. The PMMR, archived reports, and additional information is available at: https://www1.nyc.gov/site/operations/performance/mmr.page and https://dmmr.nyc.gov/nvzu-6t9yCity GovernmentThu, 07 Sep 2023 16:21:02 +0000Preliminary Mayor's Management Report Spending and Budgethttps://data.cityofnewyork.us/d/w4ne-9ea9Contains Spending and Budget information for City agencies, as reported in the Preliminary Mayoral Management Report (PMMR).w4ne-9ea9City GovernmentThu, 07 Sep 2023 16:03:32 +0000Citywide Public Computer Centershttps://data.cityofnewyork.us/d/sejx-2gn3A public computing center (“PCC”) is a facility offering computers with internet access. It may also offer other devices, services, and digital literacy and digital skills programming, for example. A PCC may be funded in whole, or in part, with public dollars in order to provide a public service (i.e., access to digital inclusion resources). Some PCCs are open to the general public while others may have membership criteria such as age. As an inventory, and as feasible, OTI’s purpose was to collect, count, and map where and how the City supports this kind of public access to the internet.sejx-2gn3Social ServicesThu, 24 Aug 2023 13:26:58 +0000Dewatered Solids and Biosolids Allocations per Biosolids Management Practice Sitehttps://data.cityofnewyork.us/d/95kh-h6zcThis dataset contains annual allocations of dewatered solids or biosolids from a specific dewatering facility to a specific management practice site in dry metric tons. 'Dewatered solids' are the solids remaining in digested sludge after removal of water in a dewatering facility. 'Biosolids', per the US Environmental Protection Agency (EPA), is treated sewage sludge that meets the EPA pollutant, pathogen & vector attraction requirements for land application and surface disposal, and the DEP has certified it with the EPA as such. Biosolids in this case are also dewatered. Depending on the material being managed (i.e., dewatered solids or biosolids), a 'management practice site' may be one of the following: 1) a third party vendor treatment facility (e.g., composting, dryer, alkaline treatment) that converts the dewatered solids into biosolids for beneficial reuse in various land application practices (e.g., agricultural land application, reclamation site application) or for distribution/marketing as a soil amendment or fertilizer; 2) a direct land application site for agricultural use (typically a farm); or 3) a landfill. The dataset includes the pathogen (disease-causing organism) reduction and vector (an animal capable of transmitting disease such as flies or rodents) attraction reduction options employed as part of the treatment and the type of biosolids class produced, if applicable. There are different biosolids class types, each having specified treatment requirements for pollutants, pathogens and vector attraction reduction, as well as general requirements and management practices, For more information regarding biosolids compliance, treatment, uses and class types please refer to: <a href="https://www.epa.gov/biosolids">https://www.epa.gov/biosolids</a>.95kh-h6zcEnvironmentTue, 22 Aug 2023 13:17:14 +0000Taxi and For Hire Vehicle (FHV) Relief Standshttps://data.cityofnewyork.us/d/vqu8-xef9This data details the location of the Taxi and For Hire Vehicle(FHV) stands and the number of spaces provided. Taxi and FHV relief stands allow drivers to park their vehicles for up to one hour. This gives drivers the opportunity to leave their vehicles and take care of personal needs. <b>TNOTE:</b> Taxi and FHV relief stands should not be confused with taxi stands. Taxi stands allow drivers to wait in their cars to pick up passengers.vqu8-xef9TransportationTue, 15 Aug 2023 17:52:25 +0000 \ No newline at end of file +New datasets from opendata.cityofnewyork.ushttps://github.com/palewire/nyc-open-data-monitorThe latest datasets posted to New York City's data portalhttp://www.rssboard.org/rss-specificationpython-feedgenTue, 28 Nov 2023 12:08:40 +0000Local Law 83 - City Agency Advertising Spendhttps://data.cityofnewyork.us/d/n56u-3hzvPer Local Law 83 of 2021, the Mayor's Office of Ethnic and Community Media is required to report annually on each agency's full advertising spend across all media categories, including ethnic and community (ECM), mainstream, out-of-home, social media, etc. This dataset reflects the raw data that MOECM received from City Agencies on their annual advertising spend. For more information, please visit the <a href="https://www.nyc.gov/site/ethnicmedia/index.page">MOECM website</a>.n56u-3hzvCity GovernmentFri, 24 Nov 2023 17:21:13 +0000Parking Violations Issued - Fiscal Year 2023https://data.cityofnewyork.us/d/869v-vr48This dataset provides data on Parking Violations Issued between July 1, 2022 to June 30, 2023. In New York City, the fiscal year begins on July 1st of one calendar year and ends on June 30th of the following calendar year. Click here to find out more about the NYC Fiscal Year. Parking Violations Issuance datasets contain violations issued during the respective fiscal year. The Issuance datasets are not updated to reflect violation status, the information only represents the violation(s) at the time they are issued. Since appearing on an issuance dataset, a violation may have been paid, dismissed via a hearing, statutorily expired, or had other changes to its status. To see the current status of outstanding parking violations, please look at the Open Parking & Camera Violations dataset. <b>NOTE:</b> This dataset includes the records from the first month of FY2024. You may find all of the Parking Violations Issued since fiscal year 2014 in this dataset collection.869v-vr48City GovernmentTue, 14 Nov 2023 14:55:34 +0000Taxi and For Hire Vehicle (FHV) Relief Stands (Map)https://data.cityofnewyork.us/d/udk9-t2yfThis data details the location of the Taxi and For Hire Vehicle(FHV) stands and the number of spaces provided. Taxi and FHV relief stands allow drivers to park their vehicles for up to one hour. This gives drivers the opportunity to leave their vehicles and take care of personal needs. <b>TNOTE:</b> Taxi and FHV relief stands should not be confused with taxi stands. Taxi stands allow drivers to wait in their cars to pick up passengers.udk9-t2yfTransportationFri, 27 Oct 2023 19:11:50 +0000DOB Safety Violationshttps://data.cityofnewyork.us/d/855j-jadyThis data set includes violations issued on devices through the New York City Department of Buildings' DOB NOW: Safety Violations module. The data is collected because the Department of Buildings tracks violation issuance and related information. This data include items such as violations number, violations type, violation issuance data, and BIN.855j-jadyHousing & DevelopmentWed, 18 Oct 2023 16:22:08 +0000Open Streets Locations Maphttps://data.cityofnewyork.us/d/6vys-sfk5New York City's Open Streets program transforms streets into public spaces open to all. These public spaces allows a variety of activities including education and cultural programming as well as building communities. For more information please visit: https://www.nyc.gov/html/dot/html/pedestrians/openstreets.shtml Please note: there may be a delay in updates to the Open Data files.6vys-sfk5HealthTue, 17 Oct 2023 19:18:56 +0000Mayor's Management Report Spending and Budgethttps://data.cityofnewyork.us/d/2jrp-puwzContains Spending and Budget information for City agencies, as reported in the Mayoral Management Report (MMR).2jrp-puwzCity GovernmentMon, 25 Sep 2023 14:06:18 +0000Mayor's Management Report Agency Resourceshttps://data.cityofnewyork.us/d/4qmi-txnkIncludes NYC agency resources from the Mayor's Management Report (MMR), such as expenditures (includes all funds), personnel, revenue and paid overtime. This data is an overview of the financial and workforce resources used by an agency and the planned resources available to the agency in a given reporting fiscal year or future fiscal years. The MMR, archived reports, and additional information is available at: https://www1.nyc.gov/site/operations/performance/mmr.page and https://dmmr.nyc.gov/4qmi-txnkCity GovernmentFri, 22 Sep 2023 17:03:49 +0000Building Elevation and Subgrade (BES) - Maphttps://data.cityofnewyork.us/d/nmzg-6q5gThe Building Elevation and Subgrade data contains New York City building centroids derived from the Department of Building's (DOB) February 26th, 2022 building footprint dataset. Each record contains a grade and first floor measurement for each building (recorded as feet above sea-level in the NADV88 vertical datum) and indicates if subgrade space exists. DCP contracted with an external data vendor to generate a single point, or centroid, that represented the location of the center of every building recorded in the DOB dataset. The dataset excluded the footprints of small accessory buildings such as sheds. Each row within the dataset represents one building centroid, and records the X and Y coordinates of that centroid in the NAD 1983 coordinate system.nmzg-6q5gCity GovernmentTue, 19 Sep 2023 20:47:20 +00002023 Open Data Plan: Civic Engagement Activitieshttps://data.cityofnewyork.us/d/xw44-6wmwReleased as a part of the 2023 Open Data Plan, this is a list of civic engagement activities reported by city agencies for the purpose of providing context, spreading awareness and increasing the use of public city data. Details include activity descriptions, date of completion and resource links to further information on the reported activity.xw44-6wmwCity GovernmentTue, 12 Sep 2023 21:27:08 +0000Building Elevation and Subgrade (BES)https://data.cityofnewyork.us/d/bsin-59hvThe Building Elevation and Subgrade data contains New York City building centroids derived from the Department of Building's (DOB) February 26th, 2022 building footprint dataset. Each record contains a grade and first floor measurement for each building (recorded as feet above sea-level in the NADV88 vertical datum) and indicates if subgrade space exists. DCP contracted with an external data vendor to generate a single point, or centroid, that represented the location of the center of every building recorded in the DOB dataset. The dataset excluded the footprints of small accessory buildings such as sheds. Each row within the dataset represents one building centroid, and records the X and Y coordinates of that centroid in the NAD 1983 coordinate system.bsin-59hvCity GovernmentTue, 12 Sep 2023 19:06:08 +00002023 Open Data Plan: Removalshttps://data.cityofnewyork.us/d/bhst-9d3fThe list of the datasets that were removed from the Open Data Plan.bhst-9d3fCity GovernmentMon, 11 Sep 2023 23:33:38 +00002023 Open Data Plan: FOIL Datasetshttps://data.cityofnewyork.us/d/93kn-fhqtThe collection of datasets identified via FOIL reporting and their respective release statuses as a part of the 2023 Open Data Plan.93kn-fhqtCity GovernmentMon, 11 Sep 2023 17:42:23 +00002023 Open Data Plan: Future Releaseshttps://data.cityofnewyork.us/d/59qh-uc8yThe collection of datasets set to be released on the Open Data Portal, according to the 2023 Open Data Plan.59qh-uc8yCity GovernmentMon, 11 Sep 2023 17:26:03 +00002023 Open Data Plan: FOIL Metricshttps://data.cityofnewyork.us/d/33mm-wzxhVarious FOIL reporting metrics as part of the 2023 Open Data Compliance Plan.33mm-wzxhCity GovernmentMon, 11 Sep 2023 16:55:54 +00002023 Open Data Plan: Website Datahttps://data.cityofnewyork.us/d/rzp5-dkkiData on public websites maintained by or on behalf of the city agencies.rzp5-dkkiCity GovernmentMon, 11 Sep 2023 15:44:31 +0000Preliminary Mayor's Management Report Agency Resourceshttps://data.cityofnewyork.us/d/nvzu-6t9yIncludes NYC agency resources from the Preliminary Mayor's Management Report (PMMR), such as expenditures (includes all funds), personnel, revenue and paid overtime. This data is an overview of the financial and workforce resources used by an agency and the planned resources available to the agency in a given reporting fiscal year or future fiscal years. The PMMR, archived reports, and additional information is available at: https://www1.nyc.gov/site/operations/performance/mmr.page and https://dmmr.nyc.gov/nvzu-6t9yCity GovernmentThu, 07 Sep 2023 16:21:02 +0000Preliminary Mayor's Management Report Spending and Budgethttps://data.cityofnewyork.us/d/w4ne-9ea9Contains Spending and Budget information for City agencies, as reported in the Preliminary Mayoral Management Report (PMMR).w4ne-9ea9City GovernmentThu, 07 Sep 2023 16:03:32 +0000Citywide Public Computer Centershttps://data.cityofnewyork.us/d/sejx-2gn3A public computing center (“PCC”) is a facility offering computers with internet access. It may also offer other devices, services, and digital literacy and digital skills programming, for example. A PCC may be funded in whole, or in part, with public dollars in order to provide a public service (i.e., access to digital inclusion resources). Some PCCs are open to the general public while others may have membership criteria such as age. As an inventory, and as feasible, OTI’s purpose was to collect, count, and map where and how the City supports this kind of public access to the internet.sejx-2gn3Social ServicesThu, 24 Aug 2023 13:26:58 +0000Dewatered Solids and Biosolids Allocations per Biosolids Management Practice Sitehttps://data.cityofnewyork.us/d/95kh-h6zcThis dataset contains annual allocations of dewatered solids or biosolids from a specific dewatering facility to a specific management practice site in dry metric tons. 'Dewatered solids' are the solids remaining in digested sludge after removal of water in a dewatering facility. 'Biosolids', per the US Environmental Protection Agency (EPA), is treated sewage sludge that meets the EPA pollutant, pathogen & vector attraction requirements for land application and surface disposal, and the DEP has certified it with the EPA as such. Biosolids in this case are also dewatered. Depending on the material being managed (i.e., dewatered solids or biosolids), a 'management practice site' may be one of the following: 1) a third party vendor treatment facility (e.g., composting, dryer, alkaline treatment) that converts the dewatered solids into biosolids for beneficial reuse in various land application practices (e.g., agricultural land application, reclamation site application) or for distribution/marketing as a soil amendment or fertilizer; 2) a direct land application site for agricultural use (typically a farm); or 3) a landfill. The dataset includes the pathogen (disease-causing organism) reduction and vector (an animal capable of transmitting disease such as flies or rodents) attraction reduction options employed as part of the treatment and the type of biosolids class produced, if applicable. There are different biosolids class types, each having specified treatment requirements for pollutants, pathogens and vector attraction reduction, as well as general requirements and management practices, For more information regarding biosolids compliance, treatment, uses and class types please refer to: <a href="https://www.epa.gov/biosolids">https://www.epa.gov/biosolids</a>.95kh-h6zcEnvironmentTue, 22 Aug 2023 13:17:14 +0000Taxi and For Hire Vehicle (FHV) Relief Standshttps://data.cityofnewyork.us/d/vqu8-xef9This data details the location of the Taxi and For Hire Vehicle(FHV) stands and the number of spaces provided. Taxi and FHV relief stands allow drivers to park their vehicles for up to one hour. This gives drivers the opportunity to leave their vehicles and take care of personal needs. <b>TNOTE:</b> Taxi and FHV relief stands should not be confused with taxi stands. Taxi stands allow drivers to wait in their cars to pick up passengers.vqu8-xef9TransportationTue, 15 Aug 2023 17:52:25 +0000 \ No newline at end of file diff --git a/data/clean/new.csv b/data/clean/new.csv index dc0d6d5..c613a0b 100644 --- a/data/clean/new.csv +++ b/data/clean/new.csv @@ -1,2 +1 @@ scrape_date,id,name,type,update_date,creation_date,creator,permalink,category,description -2023-11-27 19:13:50.009906-05:00,n56u-3hzv,Local Law 83 - City Agency Advertising Spend,dataset,2023-11-27 21:25:08+00:00,2023-11-24 17:21:13+00:00,NYC OpenData,https://data.cityofnewyork.us/d/n56u-3hzv,City Government,"Per Local Law 83 of 2021, the Mayor's Office of Ethnic and Community Media is required to report annually on each agency's full advertising spend across all media categories, including ethnic and community (ECM), mainstream, out-of-home, social media, etc. This dataset reflects the raw data that MOECM received from City Agencies on their annual advertising spend. For more information, please visit the MOECM website." diff --git a/data/clean/observations.csv b/data/clean/observations.csv index e8ea1df..db24c45 100644 --- a/data/clean/observations.csv +++ b/data/clean/observations.csv @@ -34,3 +34,4 @@ scrape_date,n 2023-11-26 19:14:01.971637-05:00,3464 2023-11-27 07:05:39.528546-05:00,3464 2023-11-27 19:13:50.009906-05:00,3465 +2023-11-28 07:05:29.688795-05:00,3465 diff --git a/data/raw/2023-11-28T07:05:29.688795-05:00.json.gz b/data/raw/2023-11-28T07:05:29.688795-05:00.json.gz new file mode 100644 index 0000000..364e23d Binary files /dev/null and b/data/raw/2023-11-28T07:05:29.688795-05:00.json.gz differ diff --git a/data/raw/latest.json.gz b/data/raw/latest.json.gz index a395111..a3bc468 100644 Binary files a/data/raw/latest.json.gz and b/data/raw/latest.json.gz differ diff --git a/data/timestamp.txt b/data/timestamp.txt index 76c1c76..36aed16 100644 --- a/data/timestamp.txt +++ b/data/timestamp.txt @@ -1 +1 @@ -Tue Nov 28 00:17:02 UTC 2023 +Tue Nov 28 12:08:42 UTC 2023