Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Send data to database using plain SQL and psycopg2 #7

Open
wants to merge 12 commits into
base: master
Choose a base branch
from

Conversation

emiedmee
Copy link

@emiedmee emiedmee commented Apr 25, 2023

The following changes since commit f0f5fa8:

Rearranging imports and adding note about overriding print() (2022-03-25 11:17:58 -0600)

are available in the Git repository at:

https://github.com/emiedmee/nmea_data_convert send-data-to-db

for you to fetch changes up to 97f3a12:

Remove sqlalchemy, only psycopg2 is needed for SQL now and comment out original send_data_to_db() (2023-04-20 19:28:59 +0200)

When testing out your program, I ran into some error ("... multiple schema ...") when inserting data into the database.
I was not able to find a fix for this error, so I decided to make a fix for this by sending data to the database using only plain SQL and psycopg2.


Emiel Demeestere (12):

  • sort imports
  • format code
  • Add types everywhere
  • Just specify datatypes as strings and create tables before insert
  • Create own function to send data to db (just the skeleton)
  • Create table if not exist and beginning for insert command
  • Only import required things
  • Some test data for merged GSA and GSV
  • Fix some column_casting data types
  • Insert dataframes in database using psycopg2 and template SQL statement
  • Replace pd.NaN and pd.NaT with None, because they are incompatible with psycopg2
  • Remove sqlalchemy, only psycopg2 is needed for SQL now and comment out original send_data_to_db()

Signed-off-by: Emiel Demeestere emiel.demeestere@ugent.be

@emiedmee emiedmee force-pushed the send-data-to-db branch 2 times, most recently from 9af4d5f to 97f3a12 Compare April 25, 2023 22:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant