Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
7896b2c
chore: Update folder name
abdulrafey38 Nov 6, 2024
93031a2
chore: update package name
abdulrafey38 Nov 7, 2024
745805e
chore: add generate_date_range func
kmaepa Nov 7, 2024
b411651
chore: change version to 1
abdulrafey38 Nov 7, 2024
8296e5e
chore: remove deacticated user as author
abdulrafey38 Nov 7, 2024
980d4f5
refactor: refactored common.py file
usama101 Nov 7, 2024
9cddbd2
refactor: refactored sitemap.py
usama101 Nov 7, 2024
3ae8e96
refactor: fix sitemap refactoring
usama101 Nov 7, 2024
68591db
fix: remove prefect references from edx api class
usama101 Nov 7, 2024
fd922c7
refactor: refactor writing report to s3 func
usama101 Nov 7, 2024
3770cf9
chore: fixes and remove task decorator from load_s3_data_to_snowflake
usama101 Nov 8, 2024
0ed9a5f
refactor: refactored s3 funcs
usama101 Nov 8, 2024
8ea7c41
chore: update paypal to remove prefect
abdulrafey38 Nov 11, 2024
bde093b
chore: bump version 1.0.10
abdulrafey38 Nov 11, 2024
177b50c
chore: update load_s3_data_to_snowflake to remove old approach
abdulrafey38 Nov 11, 2024
2126aef
chore: Remove prefect from export_snowflake_table_to_s3
abdulrafey38 Nov 19, 2024
23e60e6
chore: bump version
abdulrafey38 Nov 19, 2024
84e1566
chore: update mysql functions
abdulrafey38 Nov 19, 2024
ad5a22b
fix: fixed mysql username cred
usama101 Nov 25, 2024
9cc9fbc
fix: fixed mysql username cred (#8)
usama101 Nov 25, 2024
f652cde
chore: update export_snowflake_table_to_s3 to handle aws_credentials
abdulrafey38 Nov 29, 2024
bf2c5fa
chore: bump version
abdulrafey38 Nov 29, 2024
7b1013d
chore: added functionality for multiple files upload to snowflake
usama101 Dec 6, 2024
53be214
fix: fix variable name in snowflake.py
usama101 Dec 6, 2024
0329e4d
chore: return all objects in list_object_keys_from_s3 function
usama101 Jan 24, 2025
3f644fe
chore: incremented version
usama101 Jan 24, 2025
f524d52
chore: remove task from GA function
abdulrafey38 Jan 27, 2025
377e229
chore: remove unused import
abdulrafey38 Jan 27, 2025
bf9f7c8
chore: add test_generate_date_range unit test
kmaepa Jan 30, 2025
cee832f
chore: add unit test test_generate_date_range
kmaepa Jan 30, 2025
449b461
chore: add unit test for get_filename_safe_course_id
kmaepa Jan 30, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .coveragerc
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
[run]
branch = True
data_file = .coverage
source=edx_prefectutils
source=edx_argoutils
2 changes: 1 addition & 1 deletion .github/ISSUE_TEMPLATE.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
* edx-prefectutils version:
* edx-argoutils version:
* Python version:
* Operating System:

Expand Down
2 changes: 1 addition & 1 deletion AUTHORS.rst
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ Credits
Development Lead
----------------

* Julia Eskew <jeskew@edx.org>
* Abdul Rafey <arafey@2u.com>

Contributors
------------
Expand Down
24 changes: 12 additions & 12 deletions CONTRIBUTING.rst
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ Types of Contributions
Report Bugs
~~~~~~~~~~~

Report bugs at https://github.com/edx/edx-prefectutils/issues.
Report bugs at https://github.com/2uinc/edx-argoutils/issues.

If you are reporting a bug, please include:

Expand All @@ -38,14 +38,14 @@ and "help wanted" is open to whoever wants to implement it.
Write Documentation
~~~~~~~~~~~~~~~~~~~

edx-prefectutils could always use more documentation, whether as part of the
official edx-prefectutils docs, in docstrings, or even on the web in blog posts,
edx-argoutils could always use more documentation, whether as part of the
official edx-argoutils docs, in docstrings, or even on the web in blog posts,
articles, and such.

Submit Feedback
~~~~~~~~~~~~~~~

The best way to send feedback is to file an issue at https://github.com/edx/edx-prefectutils/issues.
The best way to send feedback is to file an issue at https://github.com/2uinc/edx-argoutils/issues.

If you are proposing a feature:

Expand All @@ -57,17 +57,17 @@ If you are proposing a feature:
Get Started!
------------

Ready to contribute? Here's how to set up `edx-prefectutils` for local development.
Ready to contribute? Here's how to set up `edx-argoutils` for local development.

1. Fork the `edx-prefectutils` repo on GitHub.
1. Fork the `edx-argoutils` repo on GitHub.
2. Clone your fork locally::

$ git clone git@github.com:your_name_here/edx-prefectutils.git
$ git clone git@github.com:your_name_here/edx-argoutils.git

3. Install your local copy into a virtualenv. Assuming you have virtualenvwrapper installed, this is how you set up your fork for local development::

$ mkvirtualenv edx-prefectutils
$ cd edx-prefectutils/
$ mkvirtualenv edx-argoutils
$ cd edx-argoutils/
$ python setup.py develop

4. Create a branch for local development::
Expand All @@ -79,7 +79,7 @@ Ready to contribute? Here's how to set up `edx-prefectutils` for local developme
5. When you're done making changes, check that your changes pass flake8 and the
tests, including testing other Python versions with tox::

$ flake8 edx_prefectutils tests
$ flake8 edx_argoutils tests
$ python setup.py test or pytest
$ tox

Expand All @@ -103,15 +103,15 @@ Before you submit a pull request, check that it meets these guidelines:
your new functionality into a function with a docstring, and add the
feature to the list in README.rst.
3. The pull request should work for Python 3.5, 3.6, 3.7 and 3.8, and for PyPy. Check
https://github.com/edx/edx-prefectutils/actions?query=workflow%3A%22Python+CI%22
https://github.com/2uinc/edx-argoutils/actions?query=workflow%3A%22Python+CI%22
and make sure that the tests pass for all supported Python versions.

Tips
----

To run a subset of tests::

$ pytest tests.test_edx_prefectutils
$ pytest tests.test_edx_argoutils


Deploying
Expand Down
2 changes: 1 addition & 1 deletion HISTORY.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
History
=======

0.1.0 (2020-05-11)
0.1.0 (2024-11-07)
------------------

* First release on PyPI.
8 changes: 4 additions & 4 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ clean-test: ## remove test and coverage artifacts
rm -fr .pytest_cache

lint: ## check style with flake8
flake8 edx_prefectutils tests
flake8 edx_argoutils tests

test: ## run tests quickly with the default Python
pytest
Expand All @@ -57,15 +57,15 @@ test-all: ## run tests on every Python version with tox
tox

coverage: ## check code coverage quickly with the default Python
coverage run --source edx_prefectutils -m pytest
coverage run --source edx_argoutils -m pytest
coverage report -m
coverage html
$(BROWSER) htmlcov/index.html

docs: ## generate Sphinx HTML documentation, including API docs
rm -f docs/edx_prefectutils.rst
rm -f docs/edx_argoutils.rst
rm -f docs/modules.rst
sphinx-apidoc -o docs/ edx_prefectutils
sphinx-apidoc -o docs/ edx_argoutils
$(MAKE) -C docs clean
$(MAKE) -C docs html
$(BROWSER) docs/_build/html/index.html
Expand Down
13 changes: 5 additions & 8 deletions README.rst
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
================
edx-prefectutils
edx-argoutils
================






A utility code written by edX specifically for edX Prefect Flows. It assists writing in edX Prefect Flows.
A utility code written by edX specifically for edX Argo Flows. It assists writing in edX Argo Flows.



Expand All @@ -20,14 +20,11 @@ PyPI Package Release
--------------------
- This repository publishes packages on PyPI, when you update this repository make sure to create a new release so latest changes would be published on PyPI and become available for use.
- Bump the `version`_ to match the version that will be released. Once the pull request with the updated version is merged into master, create a new release from GitHub Web UI, using the same version number. You can refer to `How to create GitHub release`_ for more information on this process.
- `edx-prefectutils PyPI`_ package is used in `prefect-flows`_ repository, you will have to update edx-prefectutils package version in `requirements`_ so it will pick latest PyPI package version.

.. _`version`: https://github.com/edx/edx-prefectutils/blob/master/edx_prefectutils/__init__.py#L5
.. _`How to create GitHub release`: https://docs.github.com/en/github/administering-a-repository/releasing-projects-on-github/managing-releases-in-a-repository
.. _`edx-prefectutils PyPI`: https://pypi.org/project/edx-prefectutils/
.. _`prefect-flows`: https://github.com/edx/prefect-flows
.. _`requirements`: https://github.com/edx/prefect-flows/blob/master/requirements.txt#L7

.. _`version`: https://github.com/2uinc/edx-argoutils/blob/master/edx_argoutils/__init__.py#L5
.. _`How to create GitHub release`: https://docs.github.com/en/github/administering-a-repository/releasing-projects-on-github/managing-releases-in-a-repository
.. _`edx-argoutils PyPI`: https://pypi.org/project/edx-argoutils/


Credits
Expand Down
2 changes: 1 addition & 1 deletion docs/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
# You can set these variables from the command line.
SPHINXOPTS =
SPHINXBUILD = python -msphinx
SPHINXPROJ = edx_prefectutils
SPHINXPROJ = edx_argoutils
SOURCEDIR = .
BUILDDIR = _build

Expand Down
32 changes: 16 additions & 16 deletions docs/conf.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
#!/usr/bin/env python
#
# edx_prefectutils documentation build configuration file, created by
# edx_argoutils documentation build configuration file, created by
# sphinx-quickstart on Fri Jun 9 13:47:02 2017.
#
# This file is execfile()d with the current directory set to its
Expand All @@ -21,7 +21,7 @@
import sys
sys.path.insert(0, os.path.abspath('..'))

import edx_prefectutils
import edx_argoutils

# -- General configuration ---------------------------------------------

Expand All @@ -46,18 +46,18 @@
top_level_doc = 'index'

# General information about the project.
project = 'edx-prefectutils'
copyright = "2020, Julia Eskew"
author = "Julia Eskew"
project = 'edx-argoutils'
copyright = "2024, Abdul Rafey"
author = "Abdul Rafey"

# The version info for the project you're documenting, acts as replacement
# for |version| and |release|, also used in various other places throughout
# the built documents.
#
# The short X.Y version.
version = edx_prefectutils.__version__
version = edx_argoutils.__version__
# The full version, including alpha/beta/rc tags.
release = edx_prefectutils.__version__
release = edx_argoutils.__version__

# The language for content autogenerated by Sphinx. Refer to documentation
# for a list of supported languages.
Expand Down Expand Up @@ -100,7 +100,7 @@
# -- Options for HTMLHelp output ---------------------------------------

# Output file base name for HTML help builder.
htmlhelp_basename = 'edx_prefectutilsdoc'
htmlhelp_basename = 'edx_argoutilsdoc'


# -- Options for LaTeX output ------------------------------------------
Expand All @@ -127,9 +127,9 @@
# (source start file, target name, title, author, documentclass
# [howto, manual, or own class]).
latex_documents = [
(top_level_doc, 'edx_prefectutils.tex',
'edx-prefectutils Documentation',
'Julia Eskew', 'manual'),
(top_level_doc, 'edx_argoutils.tex',
'edx-argoutils Documentation',
'Abdul Rafey', 'manual'),
]


Expand All @@ -138,8 +138,8 @@
# One entry per manual page. List of tuples
# (source start file, name, description, authors, manual section).
man_pages = [
(top_level_doc, 'edx_prefectutils',
'edx-prefectutils Documentation',
(top_level_doc, 'edx_argoutils',
'edx-argoutils Documentation',
[author], 1)
]

Expand All @@ -150,10 +150,10 @@
# (source start file, target name, title, author,
# dir menu entry, description, category)
texinfo_documents = [
(top_level_doc, 'edx_prefectutils',
'edx-prefectutils Documentation',
(top_level_doc, 'edx_argoutils',
'edx-argoutils Documentation',
author,
'edx_prefectutils',
'edx_argoutils',
'One line description of project.',
'Miscellaneous'),
]
Expand Down
2 changes: 1 addition & 1 deletion docs/index.rst
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
Welcome to edx-prefectutils's documentation!
Welcome to edx-argoutils's documentation!
=========================================

.. toctree::
Expand Down
16 changes: 8 additions & 8 deletions docs/installation.rst
Original file line number Diff line number Diff line change
Expand Up @@ -8,13 +8,13 @@ Installation
Stable release
--------------

To install edx-prefectutils, run this command in your terminal:
To install edx-argoutils, run this command in your terminal:

.. code-block:: console

$ pip install edx_prefectutils
$ pip install edx_argoutils

This is the preferred method to install edx-prefectutils, as it will always install the most recent stable release.
This is the preferred method to install edx-argoutils, as it will always install the most recent stable release.

If you don't have `pip`_ installed, this `Python installation guide`_ can guide
you through the process.
Expand All @@ -26,19 +26,19 @@ you through the process.
From sources
------------

The sources for edx-prefectutils can be downloaded from the `Github repo`_.
The sources for edx-argoutils can be downloaded from the `Github repo`_.

You can either clone the public repository:

.. code-block:: console

$ git clone git://github.com/edx/edx-prefectutils
$ git clone git://github.com/2uinc/edx-argoutils

Or download the `tarball`_:

.. code-block:: console

$ curl -OJL https://github.com/edx/edx-prefectutils/tarball/master
$ curl -OJL https://github.com/2uinc/edx-argoutils/tarball/master

Once you have a copy of the source, you can install it with:

Expand All @@ -47,5 +47,5 @@ Once you have a copy of the source, you can install it with:
$ python setup.py install


.. _Github repo: https://github.com/edx/edx-prefectutils
.. _tarball: https://github.com/edx/edx-prefectutils/tarball/master
.. _Github repo: https://github.com/2uinc/edx-argoutils
.. _tarball: https://github.com/2uinc/edx-argoutils/tarball/master
2 changes: 1 addition & 1 deletion docs/make.bat
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ if "%SPHINXBUILD%" == "" (
)
set SOURCEDIR=.
set BUILDDIR=_build
set SPHINXPROJ=edx_prefectutils
set SPHINXPROJ=edx_argoutils

if "%1" == "" goto help

Expand Down
4 changes: 2 additions & 2 deletions docs/usage.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,6 @@
Usage
=====

To use edx-prefectutils in a project::
To use edx-argoutils in a project::

import edx_prefectutils
import edx_argoutils
5 changes: 5 additions & 0 deletions edx_argoutils/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
"""
Top-level package for edx-argoutils.
"""

__version__ = '1.0.19'
File renamed without changes.
Loading
Loading