Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sqlconnector methods #15

Merged
merged 5 commits into from
Jan 7, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
76 changes: 39 additions & 37 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,77 +12,79 @@
<a href="https://github.com/psf/black"><img alt="Code style: black" src="https://img.shields.io/badge/code%20style-black-000000.svg"></a>
</p>

**SQLconnect** is a Python package designed to simplify the process of connecting to SQL databases. It uses a `sqlconnect.yaml` file for database configuration and a `sqlconnect.env` file for secure credentials management. SQLconnect supports executing SQL from `.sql` files or Python strings, and retrieving data into pandas DataFrames. This package is particularly useful for data analysts and developers who need a straightforward way to interact with SQL databases.
**SQLconnect** is a Python package designed to provide a straightforward way to interact with SQL databases (Postgres, Microsoft SQL Server, Oracle ect.). It enables direct population of DataFrames from .sql files. A `sqlconnect.yaml` file is used for database configuration and a `sqlconnect.env` file for secure credentials management.

## Features

- Allows easy configuration and management of database connections using a `sqlconnect.yaml` file.
- Turn SQL queries into DataFrames in as few as 3 lines of code

- Supports the use of a `sqlconnect.env` file for secure storage of database credentials, enhancing security.
- Easy management of multiple database connections using a configuration file

- Leverages SQLAlchemy for database connections, providing a robust and flexible framework for SQL operations.
- Secure storage of database credentials using environment variables

- Enables executing SQL queries and directly retrieving the results into pandas DataFrames, facilitating easy data manipulation and analysis.
- Execute SQL queries and commands directly from .sql files or from a string

- Capable of handling multiple database connections, allowing users to switch between different databases as needed.
- Integration with SQLAlchemy & Pandas providing robust and flexible SQL operations

- Provides functionality to execute SQL queries either by reading from an external file or directly from a string.
```python
import sqlconnect as sc

## Installation
connection = sc.Sqlconnector("My_Database") # Set up a database connection based on sqlconnect.yaml

```bash
pip install sqlconnect
df = connection.sql_to_df("path/to/sql_query.sql") # Assign the results of a query to a DataFrame

print(df.describe()) # Explore the DataFrame with Pandas
```


## Configuration

To use SQLconnect, create a `sqlconnect.yaml` file in the root of your project (or in your home directory) with the following example structure:

```yaml
connections:
Database_PROD:
sqlalchemy_driver: 'mssql+pyodbc'
odbc_driver: 'SQL+Server'
server: 'prod-server.database.com'
database: 'ProdDB'
username: '${DB_PROD_USERNAME}' # References DB_PROD_USERNAME in sqlconnect.env
password: '${DB_PROD_PASSWORD}' # References DB_PROD_PASSWORD in sqlconnect.env
options:
- 'Trusted_Connection=No'
My_Database:
dialect: 'mssql'
dbapi: 'pyodbc'
host: 'prod-server.example.com'
database: 'master'
username: '${MSSQL_USERNAME}' # References MSSQL_USERNAME in sqlconnect.env
password: '${MSSQL_PASSWORD}' # References MSSQL_PASSWORD in sqlconnect.env
options:
driver: 'ODBC Driver 17 for SQL Server'

A_Postgres_Database:
dialect: 'postgresql'
dbapi: 'psycopg2'
host: 'dbserver123.company.com'
database: 'postgres'
username: '${POSTGRES_USERNAME}'
password: '${POSTGRES_PASSWORD}'
```

Also create a `sqlconnect.env` file in the root of your project (or in your home directory) with the following example structure:

```bash
# This file should be kept secure and not checked into version control (add to .gitignore)
# Production Database Credentials
DB_PROD_USERNAME=prodUser
DB_PROD_PASSWORD=prodPassword
MSSQL_USERNAME=prodUsername
MSSQL_PASSWORD=actualprodPassword
POSTGRES_USERNAME=postgresProdUsername
POSTGRES_PASSWORD=actualprodPassword
```

Replace the example values with your actual database connection details.

## Usage

Here's a quick example to get you started:

```python
import sqlconnect as sc
## Documentation

# Set up a database connection, all configuration is handled with sqlconnect.yaml and sqlconnect.env
connection = sc.Sqlconnector("Database_PROD")
Full documentation for SQLconnect can be found at https://sqlconnect.readthedocs.io/

# Assign the results of a query to a pandas DataFrame
df = connection.sql_to_df("your_query.sql")
## Installation

# Explore the dataframe with pandas
print(df.describe())
```bash
pip install sqlconnect
```

## Documentation

Full documentation for SQLconnect can be found at https://sqlconnect.readthedocs.io/

## License

This project is licensed under the [MIT License](https://raw.githubusercontent.com/JustinFrizzell/sqlconnect/main/LICENCE).
3 changes: 2 additions & 1 deletion docs/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,8 @@


# The name of the Pygments (syntax highlighting) style to use.
pygments_style = "sphinx"
pygments_style = "default"
pygments_dark_style = "material"

# We need headers to be linkable to so ask MyST-Parser to autogenerate anchor IDs for
# headers up to and including level 3.
Expand Down
6 changes: 4 additions & 2 deletions docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,9 @@
.. raw:: html

<div style="text-align: center;">
<a href="https://github.com/JustinFrizzell/sqlconnect/actions/workflows/ci.yaml">
<img src="https://github.com/JustinFrizzell/sqlconnect/actions/workflows/ci.yaml/badge.svg" alt="CI-testing">
</a>
<a href="https://pypi.org/project/sqlconnect/">
<img src="https://img.shields.io/pypi/v/sqlconnect" alt="PyPI">
</a>
Expand All @@ -20,7 +23,7 @@
</div>
<div style="height: 20px;"></div> <!-- Spacer -->

**SQLconnect** is a Python package designed to simplify the process of connecting to SQL databases. It uses a ``sqlconnect.yaml`` file for database configuration and a ``sqlconnect.env`` file for secure credentials management. SQLconnect supports executing SQL from ``.sql`` files or Python strings, and retrieving data into pandas DataFrames. This package is particularly useful for data analysts and developers who need a straightforward way to interact with SQL databases.
**SQLconnect** is a Python package designed to provide a straightforward way to interact with SQL databases (Postgres, Microsoft SQL Server, Oracle ect.). It enables direct population of DataFrames from .sql files. A ``sqlconnect.yaml`` file is used for database configuration and a ``sqlconnect.env`` file for secure credentials management.

Contents
========
Expand All @@ -35,7 +38,6 @@ Contents
:caption: User Guide

install
tutorial
documentation

.. toctree::
Expand Down
6 changes: 3 additions & 3 deletions setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,9 @@

setuptools.setup(
name="sqlconnect",
version="0.2.1",
version="0.3.0",
author="Justin Frizzell",
description="Package to simplify connections to SQL databases.",
description="Simplifies connections to SQL databases for data analysts. Populate DataFrames with the results of queries directly from .sql files.",
long_description=Path("README.md").read_text(encoding="utf-8"),
long_description_content_type="text/markdown",
url="https://github.com/JustinFrizzell/sqlconnect",
Expand All @@ -14,7 +14,7 @@
"Source": "https://github.com/JustinFrizzell/sqlconnect",
},
packages=setuptools.find_packages(exclude=["tests", "tests.*"]),
install_requires=["pandas", "sqlalchemy", "pyyaml", "pyodbc", "python-dotenv"],
install_requires=["pandas", "sqlalchemy", "pyyaml", "python-dotenv", "pyodbc", "psycopg2-binary", "oracledb", "PyMySQL"],
classifiers=[
"License :: OSI Approved :: MIT License",
"Intended Audience :: Developers",
Expand Down
Loading