Python file created locally are converted to notebooks

[maxencedraguet-tomtom]

Hello! Really like this package, thanks!

When creating locally a python worskpace file in VS code (with ".py" extension), databricks power tools sees it as a [PYTHON] notebook (with .py extension) instead of as a [FILE].

When synchronising with Databricks, it also modifies this python file into a python Notebook, adding the # Databricks notebook source header automatically.

Is there a way to keep ".py" files as files and not convert it to a notebook?

Config:

• My export formats: exportFormats = {"Scala": ".scala","Python": ".py","SQL": ".sql","R": ".r"}

[gbrueckl]

thanks for reporting this issue - seems like a bug that we should fix

would the following work for you:

• check the local ´.py´ file
• if it stats with # Databricks notebook source -> Notebook stored as .py
• otherwise treat as [FILE] (e.g. Python library)

[maxencedraguet-tomtom]

Hello @gbrueckl. What you propose would be ideal. It would allow users to create python workspace files locally and synchronise them with Databricks.

[gbrueckl]

ok, I think I have a fix ready

but just to be sure - this only applies to new items created locally first
for existing items we always take the type (e.g. FILE or NOTEBOOK) from the Databricks service.

just published v2.3.3 to the gallery which should be available soon then

[maxencedraguet-tomtom]

Thanks for the super quick fix! I think it's fine if it only applies to new items.

[gbrueckl]

if your issue is fixed now, please close this ticket

thanks,
-gerhard

[maxencedraguet-tomtom]

Hey! Sorry to reopen this but there still is a small issue. While new workspace python files are correctly stored as files, they're not passed with their extension in the file name. For example, a __init__.py file is transformed into a __init__ file. This causes an issues when loading code from different files.

Would it be possible, for workspace python files, to keep the extension in the name?

[gbrueckl]

just published v2.3.4 which fixes this issue
please report back if you have any other issues

[maxencedraguet-tomtom]

Great thanks!