Skip to content

Commit

Permalink
fix: cassandra configs (#364)
Browse files Browse the repository at this point in the history 
* fix: to lower case
* pin numpy
  • Loading branch information
@ralphrass
ralphrass authored Jun 20, 2024
1 parent 2a5a6e8 commit 6363e03
Show file tree
Hide file tree
Showing 2 changed files with 17 additions and 14 deletions.
30 changes: 16 additions & 14 deletions butterfree/configs/db/cassandra_config.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -228,26 +228,28 @@ def translate(self, schema: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
"""
cassandra_mapping = {
"TimestampType": "timestamp",
"BinaryType": "boolean",
"BooleanType": "boolean",
"DateType": "timestamp",
"DecimalType": "decimal",
"DoubleType": "double",
"FloatType": "float",
"IntegerType": "int",
"LongType": "bigint",
"StringType": "text",
"ArrayType(LongType,true)": "frozen<list<bigint>>",
"ArrayType(StringType,true)": "frozen<list<text>>",
"ArrayType(FloatType,true)": "frozen<list<float>>",
"timestamptype": "timestamp",
"binarytype": "boolean",
"booleantype": "boolean",
"datetype": "timestamp",
"decimaltype": "decimal",
"doubletype": "double",
"floattype": "float",
"integertype": "int",
"longtype": "bigint",
"stringtype": "text",
"arraytype(longtype,true)": "frozen<list<bigint>>",
"arraytype(stringtype,true)": "frozen<list<text>>",
"arraytype(floattype,true)": "frozen<list<float>>",
}
cassandra_schema = []
for features in schema:
cassandra_schema.append(
{
"column_name": features["column_name"],
"type": cassandra_mapping[str(features["type"]).replace("()", "")],
"type": cassandra_mapping[
str(features["type"]).replace("()", "").lower()
],
"primary_key": features["primary_key"],
}
)
Expand Down
1 change: 1 addition & 0 deletions requirements.txt
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,3 +6,4 @@ pyspark==3.5.1
typer==0.4.2
typing-extensions>3.7.4,<5
boto3==1.17.*
numpy==1.26.4

1 comment on commit 6363e03

@chip-n-dale
Copy link

@chip-n-dale chip-n-dale bot commented on 6363e03 Jun 20, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @ralphrass!

The GitLeaks SecTool reported some possibly exposed credentials/secrets, how about giving them a look?

GitLeaks Alert Sync 
[
  {
    "line": "    webhook: REDACTED",
    "lineNumber": 141,
    "offender": "REDACTED",
    "offenderEntropy": -1,
    "commit": "b6a5daf28abc035f74b9685aab573d384680b9d1",
    "repo": "butterfree",
    "repoURL": "",
    "leakURL": "",
    "rule": "Slack Webhook",
    "commitMessage": "initial commit\n",
    "author": "Alvaro",
    "email": "alvaro.marques.andrade@gmail.com",
    "file": ".drone.yml",
    "date": "2020-01-03T14:21:51-03:00",
    "tags": "key, slack"
  },
  {
    "line": "    webhook: REDACTED",
    "lineNumber": 159,
    "offender": "REDACTED",
    "offenderEntropy": -1,
    "commit": "b6697aa708fec0c5a9e3af0b2713cee6f45ff675",
    "repo": "butterfree",
    "repoURL": "",
    "leakURL": "",
    "rule": "Slack Webhook",
    "commitMessage": "hail to the butterfree\n",
    "author": "Alvaro",
    "email": "alvaro.marques.andrade@gmail.com",
    "file": ".drone.yml",
    "date": "2020-01-03T11:07:44-03:00",
    "tags": "key, slack"
  }
]

In case of false-positives, more information is available on GitLeaks FAQ
If you had any other problem or question during this process, be sure to contact us on the Security space on GChat!

Please sign in to comment.