Interesting idea

[pycal]

Hey @asuiu , I think this is a very interesting idea and attractive for a project I'm managing.

I'd be interested to know if you have plans to implement the various MigrationStrategies?

Also I was taking a look at some examples and if I'm not mistaken, in 1.2.24 some are no longer working because of recent changes (unless I'm mistaken?) Can you let me know if I should be able to do this?

from sparkorm import Struct

e.g. https://github.com/asuiu/SparkORM/blob/master/examples/conferences_extended/conferences.py#L3

[asuiu]

Hi!
You are right, the from sparkorm import Struct doesn't work due to not exported Struct in the init.
I have fixed this, and the example should work now, in the version 1.2.25, uploaded on PyPi as well.

Yes, I have plans to implement more MigrationStrategies. Do you have any specific examples in mind?

[pycal]

Nice.

Yes, I have a couple concrete use cases for migration strategies right now actually!

The simplest is "add new column to existing schema."

The other immediate use is "modify existing column type, given a specified cast":

• Simple example would be "cast IntegerType to DoubleType"
• More complex example would be "cast LongType epoch in usecs to TimestampType"