Skip to content

add cast, fix order by with expressions#1702

Open
shcheklein wants to merge 1 commit intomainfrom
cast-and-fixes
Open

add cast, fix order by with expressions#1702
shcheklein wants to merge 1 commit intomainfrom
cast-and-fixes

Conversation

@shcheklein
Copy link
Copy Markdown
Contributor

Public cast API and expression order_by fixes

Summary

  • added public func.cast(...)
  • made cast usable in mutate(), filter(), order_by(), and merge()
  • fixed order_by() expression handling to match partition_by more closely
  • fixed SQLite datetime cast runtime behavior
  • added docs and broader tests

Examples

Cast in regular expressions

dc.read_values(id_str=["10", "2", "1"]).mutate(
    id_int=func.cast("id_str", int)
).order_by(func.cast("id_str", int))

Merge on mismatched key types

left.merge(
    right,
    on=func.cast(left.c("id_str"), int),
    right_on=right.c("id_int"),
    inner=True,
)

order_by with expressions

dc.read_values(id=[2, 1, 3]).order_by(dc.C("id"), descending=True)

dc.read_values(a=[10, 1, 5], b=[1, 20, 2]).order_by(
    dc.C("a") + dc.C("b"),
    descending=True,
)

dc.read_values(file=[...]).order_by(
    func.string.length(dc.C("file.path")) + dc.C("file.size")
)

Datetime cast on SQLite

from datetime import datetime

dc.read_values(ts_str=["2024-01-02 03:04:05"]).mutate(
    ts=func.cast("ts_str", datetime)
)

Fixes

  • resolve_columns() now handles string names, DataChain function expressions, and existing SQL expressions correctly for order_by()
  • SQLite datetime casts now compile through datetime(...) instead of raw CAST(... AS DATETIME)
  • DateTime read conversion now parses ISO-like strings back into Python datetime

Tests

  • functional tests for cast in mutate(), filter(), order_by(), and merge()
  • scalar runtime cast coverage for int, float, str, bool, datetime
  • unit result-type coverage for int, float, str, bool, bytes, datetime, list[int], dict[str, int]
  • expression order_by() coverage next to the existing order-by unit tests

Docs

  • documented func.cast in the function reference

@codecov
Copy link
Copy Markdown

codecov bot commented Apr 2, 2026

Codecov Report

❌ Patch coverage is 93.10345% with 2 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
src/datachain/sql/types.py 66.66% 2 Missing ⚠️

📢 Thoughts on this report? Let us know!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant