Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Excellent work #4

Open
JimCarnicelli opened this issue Jan 6, 2025 · 0 comments
Open

Excellent work #4

JimCarnicelli opened this issue Jan 6, 2025 · 0 comments

Comments

@JimCarnicelli
Copy link

Thank you for this. It works very well for me. I'm using it presently to process the US Census Department's 2020 census block shapes for each state:

https://www.census.gov/cgi-bin/geo/shapefiles/index.php?year=2024&layergroup=Blocks+%282020%29

I had to write code to fetch and unzip all the states' archives. I wanted the shapes and metadata about each. here's a look at what these blocks look like when opened in Google Earth Pro:

image

Here's the approach I'm taking to processing one state. I need one DBF, one SHX, and one SHP file:

const dbfFile = await loadFile(`${tempPath}/${sourceName}/${sourceName}.dbf`)
const shpFile = await loadFile(`${tempPath}/${sourceName}/${sourceName}.shp`)
const shxFile = await loadFile(`${tempPath}/${sourceName}/${sourceName}.shx`)
const dbfReader = await DbfReader.fromFile(dbfFile)
const shpReader = await ShapeReader.fromFile(shpFile, shxFile)
for (let i = 0; i < shpReader.recordCount; i++) {
    const dbf = dbfReader.readRecord(i)
    const geom = shpReader.readGeom(i)
    const geoJson = geom.toGeoJson() as GeoJsonPolygon
    const fields: any = {}
    dbfReader.fields.forEach((f, j) => {
        fields[f.name] = dbf[j]
    })

    // Your code for handling one census block goes here:
    console.log(fields, geoJson.coordinates)

}

The only challenge I had was in creating the loadFile() function. I used your on fileMock.ts code as a starting point:

export const loadFile = async (filePath: string): Promise<FileMock> => {
    const buf = await fs.promises.readFile(filePath)
    const ab = new ArrayBuffer(buf.length)
    const view = new Uint8Array(ab)
    for (let i = 0; i < buf.length; ++i) {
        view[i] = buf[i]
    }
    return new FileMock(ab)
}

class FileMock implements File {
    private _buf: ArrayBuffer

    constructor(buffer: ArrayBuffer) {
        this._buf = buffer
    }

    lastModified: number
    name: string
    webkitRelativePath: string
    size: number
    type: string
    bytes(): Promise<Uint8Array> {
        throw new Error('Method not implemented.')
    }
    slice(start?: number, end?: number, contentType?: string): Blob {
        throw new Error('Method not implemented.')
    }
    stream(): ReadableStream<Uint8Array> {
        throw new Error('Method not implemented.')
    }
    text(): Promise<string> {
        throw new Error('Method not implemented.')
    }

    public async arrayBuffer(): Promise<ArrayBuffer> {
        return this._buf
    }
}

No doubt I could have done this part in a more elegant way than I ultimately did. Consider adding handlers for file paths as strings if you decide to update this someday.

Thank you again for sharing!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant