Destination MSSQL - Add Typing and Deduping #37010
Replies: 22 comments 1 reply
-
|
Hi @jbfbell, 1+ vote from me. I have to use MS SQL server for work and found the normalisation with DBT (Destination V1) to be very very slow as it uses views. It might be due to the sync is based on cursors and not CDC. I hope that Typing and Deduping with Destination V2 would be much faster, but I'm not an expert. |
Beta Was this translation helpful? Give feedback.
-
|
Support for MSSQL is the most important factor why I evaluate Airbyte. In my usecase I'm fetching data from ERP system to ecommerce. |
Beta Was this translation helpful? Give feedback.
-
|
+1 on this. |
Beta Was this translation helpful? Give feedback.
-
|
+1 - this would be extremely helpful |
Beta Was this translation helpful? Give feedback.
-
|
+1 interest for sure |
Beta Was this translation helpful? Give feedback.
-
|
+1 from me, it would be awesome to have |
Beta Was this translation helpful? Give feedback.
-
|
+1000. Why does this continue to pop up, other than prioritization? The same question was asked for Clickhouse. This is obviously one of the best features of Airbyte and having to orchestrate dbt on top of syncs is a huge pain that airbyte already solves. I can understand that you have limited time to support all the connectors, but there's really no reason to go removing this feature proactively since the end user can turn it off if they want to and it's SUPER CRAZY useful in it's current form. |
Beta Was this translation helpful? Give feedback.
-
|
+1 from me too. This is an incredibly useful feature to have integrated. As others have mentioned above, I think the "typing and deduping by default with optional opt-out" is the way to go for pretty much any RDBMS. |
Beta Was this translation helpful? Give feedback.
-
|
It was working for my tables, so +1 with optional opt-out. |
Beta Was this translation helpful? Give feedback.
-
|
I just got through a 7-month project to get Airbyte approved and running in production, and just found out that the final tables are no longer created (we were using an old version of Airbyte and the MSSQL connector, so they were created in all our testing). It's not the end of the world - using JSON directly or writing dbt/Python to convert it is OKAY... just not as excellent/fun/exciting as it was. |
Beta Was this translation helpful? Give feedback.
-
|
+1. As a leading enterprise RDBMS, many will use MSSQL source and destination. In our evaluation, we found MSSQL as a source was recently certified under #34873, but overall has some limitations: |
Beta Was this translation helpful? Give feedback.
-
|
+1 for us as well |
Beta Was this translation helpful? Give feedback.
-
|
Just to add here that in my experience, this feature is an important gateway for SMEs with older stacks to move towards more modern tooling. Definitely +1 for us |
Beta Was this translation helpful? Give feedback.
-
|
This feature is critical for numerous organizations. Could you provide an update on when it will be reinstated? The absence of this functionality is preventing many organizations from upgrading to version 1.0 and is forcing them to continue running legacy versions. |
Beta Was this translation helpful? Give feedback.
-
|
totally agree |
Beta Was this translation helpful? Give feedback.
-
|
+1. This is a big blocker for us, and keeping us from rolling out our solution to multiple clients. |
Beta Was this translation helpful? Give feedback.
-
|
Could we please consider implementing this soon? |
Beta Was this translation helpful? Give feedback.
-
|
+1 Definitely need this feature 👍 |
Beta Was this translation helpful? Give feedback.
-
|
+1 for this feature, please add it back! I think there would be more feedback if this thread were more visible |
Beta Was this translation helpful? Give feedback.
-
|
Any news on this ? I would really like to see this coming in very near future. Let me know if I or my team can help. |
Beta Was this translation helpful? Give feedback.
-
|
+1 for this feature! I also think there would be more feedback if this thread were more visible |
Beta Was this translation helpful? Give feedback.
-
|
Beta Was this translation helpful? Give feedback.
-
|
Good news! [Request for Beta Testers] Destination MSSQL V2 is open for early access! #53618 |
Beta Was this translation helpful? Give feedback.
-
We removed "normalization" from Destination MSSQL. Normalization was a process which would extract data from the raw json format and cast the data into tables with the appropriate types. For other destinations, we implemented Typing and Deduping to replace normalization and we will bring typing and deduplication to MSSQL if there is sufficient interest
The purpose of this discussion is to gauge interest in bringing this functionality back to MSSQL.
Without typing and deduping, you would need to integrate another tool, such as dbt to accomplish the same result. While integrating another tool gives you greater customization over the data format, it comes with additional operational overhead.
Should Airbyte bring Typing and Deduping to MSSQL?
Beta Was this translation helpful? Give feedback.
All reactions