Can insert the same id twice into a columnar table with an auto-embedding

[sanikolaev]

Bug Description:

mysql> drop table if exists t; CREATE TABLE t (f TEXT, embedding_vector FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2' MODEL_NAME = 'sentence-transformers/all-MiniLM-L6-v2' FROM = 'f') engine='columnar'; insert into t(id,f) values(1, 'abc'); insert into t(id,f) values(1, 'abc');
--------------
drop table if exists t
--------------

Query OK, 0 rows affected (0.00 sec)

--------------
CREATE TABLE t (f TEXT, embedding_vector FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2' MODEL_NAME = 'sentence-transformers/all-MiniLM-L6-v2' FROM = 'f') engine='columnar'
--------------

Query OK, 0 rows affected (0.00 sec)

--------------
insert into t(id,f) values(1, 'abc')
--------------

Query OK, 1 row affected (0.02 sec)

--------------
insert into t(id,f) values(1, 'abc')
--------------

Query OK, 1 row affected (0.02 sec)

Expected:

ERROR 1064 (42000): duplicate id '1'

Notes:

• removing engine='columnar' fixes the issue

Manticore Search Version:

Manticore 13.9.1 adbbf4d8a@25082815 dev (columnar 8.0.3 4fd4f52@25082510) (secondary 8.0.3 4fd4f52@25082510) (knn 8.0.3 4fd4f52@25082510) (embeddings 1.0.1)

Operating System Version:

Ubuntu 21.10

Have you tried the latest development version?

Yes

Internal Checklist:

^{To be completed by the assignee. Check off tasks that have been completed or are not applicable.}

Details

• Implementation completed
• Tests developed
• Documentation updated
• Documentation reviewed

[glookka]

Fixed in manticoresoftware/manticoresearch@dc404e2
Note that this issue is completely unrelated to embeddings

[sanikolaev]

Another example w/o auto-embeddings:

mysql> drop table if exists t; CREATE TABLE t (f TEXT) engine='columnar'; insert into t(id,f) values(1, 'abc'); insert into t(id,f) values(1, 'abc'); select * from t;
--------------
drop table if exists t
--------------

Query OK, 0 rows affected (0.00 sec)

--------------
CREATE TABLE t (f TEXT) engine='columnar'
--------------

Query OK, 0 rows affected (0.00 sec)

--------------
insert into t(id,f) values(1, 'abc')
--------------

Query OK, 1 row affected (0.00 sec)

--------------
insert into t(id,f) values(1, 'abc')
--------------

Query OK, 1 row affected (0.00 sec)

--------------
select * from t
--------------

+------+------+
| id   | f    |
+------+------+
|    1 | abc  |
+------+------+
1 row in set (0.00 sec)
--- 1 out of 1 results in 0ms ---