Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] parquet_test.py::test_many_column_project test fails on GB100 and cuda12.8 #12230

Open
yinqingh opened this issue Feb 26, 2025 · 1 comment
Labels
? - Needs Triage Need team to review and classify bug Something isn't working

Comments

@yinqingh
Copy link
Collaborator

yinqingh commented Feb 26, 2025

Describe the bug
Observed this failure in the test on GB100 with cuda12.8

_______________________________ test_many_column_project _______________________________
[gw0] linux -- Python 3.10.16 /opt/conda/bin/python

    @disable_ansi_mode  # https://github.com/NVIDIA/spark-rapids/issues/5114
    def test_many_column_project():
        def _create_wide_data_frame(spark, num_cols):
            schema_dict = {}
            for i in range(num_cols):
                schema_dict[f"c{i}"] = i
            return spark.createDataFrame([Row(**r) for r in [schema_dict]])\
                .withColumn('out', f.col('c1') * 100)

>       assert_gpu_and_cpu_are_equal_collect(
            func=lambda spark: _create_wide_data_frame(spark, 1000),
            is_cpu_first=False)

../../src/main/python/parquet_test.py:853:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
../../src/main/python/asserts.py:599: in assert_gpu_and_cpu_are_equal_collect
    _assert_gpu_and_cpu_are_equal(func, 'COLLECT', conf=conf, is_cpu_first=is_cpu_first, result_canonicalize_func_before_compare=result_canonicalize_func_before_compare)
../../src/main/python/asserts.py:521: in _assert_gpu_and_cpu_are_equal
    assert_equal(from_cpu, from_gpu)
../../src/main/python/asserts.py:111: in assert_equal
    _assert_equal(cpu, gpu, float_check=get_float_check(), path=[])
../../src/main/python/asserts.py:43: in _assert_equal
    _assert_equal(cpu[index], gpu[index], float_check, path + [index])
../../src/main/python/asserts.py:36: in _assert_equal
    _assert_equal(cpu[field], gpu[field], float_check, path + [field])
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

cpu = 1, gpu = 0
float_check = <function get_float_check.<locals>.<lambda> at 0x7ff67baca830>
path = [0, 'c1']

    def _assert_equal(cpu, gpu, float_check, path):
        t = type(cpu)
        if (t is Row):
            assert len(cpu) == len(gpu), "CPU and GPU row have different lengths at {} CPU: {} GPU: {}".format(path, len(cpu), len(gpu))
            if hasattr(cpu, "__fields__") and hasattr(gpu, "__fields__"):
                assert cpu.__fields__ == gpu.__fields__, "CPU and GPU row have different fields at {} CPU: {} GPU: {}".format(path, cpu.__fields__, gpu.__fields__)
                for field in cpu.__fields__:
                    _assert_equal(cpu[field], gpu[field], float_check, path + [field])
            else:
                for index in range(len(cpu)):
                    _assert_equal(cpu[index], gpu[index], float_check, path + [index])
        elif (t is list):
            assert len(cpu) == len(gpu), "CPU and GPU list have different lengths at {} CPU: {} GPU: {}".format(path, len(cpu), len(gpu))
            for index in range(len(cpu)):
                _assert_equal(cpu[index], gpu[index], float_check, path + [index])
        elif (t is tuple):
            assert len(cpu) == len(gpu), "CPU and GPU list have different lengths at {} CPU: {} GPU: {}".format(path, len(cpu), len(gpu))
            for index in range(len(cpu)):
                _assert_equal(cpu[index], gpu[index], float_check, path + [index])
        elif (t is pytypes.GeneratorType):
            index = 0
            # generator has no zip :( so we have to do this the hard way
            done = False
            while not done:
                sub_cpu = None
                sub_gpu = None
                try:
                    sub_cpu = next(cpu)
                except StopIteration:
                    done = True

                try:
                    sub_gpu = next(gpu)
                except StopIteration:
                    done = True

                if done:
                    assert sub_cpu == sub_gpu and sub_cpu == None, "CPU and GPU generators have different lengths at {}".format(path)
                else:
                    _assert_equal(sub_cpu, sub_gpu, float_check, path + [index])

                index = index + 1
        elif (t is dict):
            # The order of key/values is not guaranteed in python dicts, nor are they guaranteed by Spark
            # so sort the items to do our best with ignoring the order of dicts
            cpu_items = list(cpu.items()).sort(key=_RowCmp)
            gpu_items = list(gpu.items()).sort(key=_RowCmp)
            _assert_equal(cpu_items, gpu_items, float_check, path + ["map"])
        elif (t is int):
>           assert cpu == gpu, "GPU and CPU int values are different at {}".format(path)
E           AssertionError: GPU and CPU int values are different at [0, 'c1']

../../src/main/python/asserts.py:78: AssertionError
--------------------------------- Captured stdout call ---------------------------------
### GPU RUN ###
### CPU RUN ###
### COLLECT: GPU TOOK 7.501291036605835 CPU TOOK 0.39443254470825195 ###
--- CPU OUTPUT
+++ GPU OUTPUT
@@ -1 +1 @@
-Row(c0=0, c1=1, c2=2, c3=3, c4=4, c5=5, c6=6, c7=7, c8=8, c9=9, c10=10, c11=11, c12=12, c13=13, c14=14, c15=15, c16=16, c17=17, c18=18, c19=19, c20=20, c21=21, c22=22, c23=23, c24=24, c25=25, c26=26, c27=27, c28=28, c29=29, c30=30, c31=31, c32=32, c33=33, c34=34, c35=35, c36=36, c37=37, c38=38, c39=39, c40=40, c41=41, c42=42, c43=43, c44=44, c45=45, c46=46, c47=47, c48=48, c49=49, c50=50, c51=51, c52=52, c53=53, c54=54, c55=55, c56=56, c57=57, c58=58, c59=59, c60=60, c61=61, c62=62, c63=63, c64=64, c65=65, c66=66, c67=67, c68=68, c69=69, c70=70, c71=71, c72=72, c73=73, c74=74, c75=75, c76=76, c77=77, c78=78, c79=79, c80=80, c81=81, c82=82, c83=83, c84=84, c85=85, c86=86, c87=87, c88=88, c89=89, c90=90, c91=91, c92=92, c93=93, c94=94, c95=95, c96=96, c97=97, c98=98, c99=99, c100=100, c101=101, c102=102, c103=103, c104=104, c105=105, c106=106, c107=107, c108=108, c109=109, c110=110, c111=111, c112=112, c113=113, c114=114, c115=115, c116=116, c117=117, c118=118, c119=119, c120=120, c121=121, c122=122, c123=123, c124=124, c125=125, c126=126, c127=127, c128=128, c129=129, c130=130, c131=131, c132=132, c133=133, c134=134, c135=135, c136=136, c137=137, c138=138, c139=139, c140=140, c141=141, c142=142, c143=143, c144=144, c145=145, c146=146, c147=147, c148=148, c149=149, c150=150, c151=151, c152=152, c153=153, c154=154, c155=155, c156=156, c157=157, c158=158, c159=159, c160=160, c161=161, c162=162, c163=163, c164=164, c165=165, c166=166, c167=167, c168=168, c169=169, c170=170, c171=171, c172=172, c173=173, c174=174, c175=175, c176=176, c177=177, c178=178, c179=179, c180=180, c181=181, c182=182, c183=183, c184=184, c185=185, c186=186, c187=187, c188=188, c189=189, c190=190, c191=191, c192=192, c193=193, c194=194, c195=195, c196=196, c197=197, c198=198, c199=199, c200=200, c201=201, c202=202, c203=203, c204=204, c205=205, c206=206, c207=207, c208=208, c209=209, c210=210, c211=211, c212=212, c213=213, c214=214, c215=215, c216=216, c217=217, c218=218, c219=219, c220=220, c221=221, c222=222, c223=223, c224=224, c225=225, c226=226, c227=227, c228=228, c229=229, c230=230, c231=231, c232=232, c233=233, c234=234, c235=235, c236=236, c237=237, c238=238, c239=239, c240=240, c241=241, c242=242, c243=243, c244=244, c245=245, c246=246, c247=247, c248=248, c249=249, c250=250, c251=251, c252=252, c253=253, c254=254, c255=255, c256=256, c257=257, c258=258, c259=259, c260=260, c261=261, c262=262, c263=263, c264=264, c265=265, c266=266, c267=267, c268=268, c269=269, c270=270, c271=271, c272=272, c273=273, c274=274, c275=275, c276=276, c277=277, c278=278, c279=279, c280=280, c281=281, c282=282, c283=283, c284=284, c285=285, c286=286, c287=287, c288=288, c289=289, c290=290, c291=291, c292=292, c293=293, c294=294, c295=295, c296=296, c297=297, c298=298, c299=299, c300=300, c301=301, c302=302, c303=303, c304=304, c305=305, c306=306, c307=307, c308=308, c309=309, c310=310, c311=311, c312=312, c313=313, c314=314, c315=315, c316=316, c317=317, c318=318, c319=319, c320=320, c321=321, c322=322, c323=323, c324=324, c325=325, c326=326, c327=327, c328=328, c329=329, c330=330, c331=331, c332=332, c333=333, c334=334, c335=335, c336=336, c337=337, c338=338, c339=339, c340=340, c341=341, c342=342, c343=343, c344=344, c345=345, c346=346, c347=347, c348=348, c349=349, c350=350, c351=351, c352=352, c353=353, c354=354, c355=355, c356=356, c357=357, c358=358, c359=359, c360=360, c361=361, c362=362, c363=363, c364=364, c365=365, c366=366, c367=367, c368=368, c369=369, c370=370, c371=371, c372=372, c373=373, c374=374, c375=375, c376=376, c377=377, c378=378, c379=379, c380=380, c381=381, c382=382, c383=383, c384=384, c385=385, c386=386, c387=387, c388=388, c389=389, c390=390, c391=391, c392=392, c393=393, c394=394, c395=395, c396=396, c397=397, c398=398, c399=399, c400=400, c401=401, c402=402, c403=403, c404=404, c405=405, c406=406, c407=407, c408=408, c409=409, c410=410, c411=411, c412=412, c413=413, c414=414, c415=415, c416=416, c417=417, c418=418, c419=419, c420=420, c421=421, c422=422, c423=423, c424=424, c425=425, c426=426, c427=427, c428=428, c429=429, c430=430, c431=431, c432=432, c433=433, c434=434, c435=435, c436=436, c437=437, c438=438, c439=439, c440=440, c441=441, c442=442, c443=443, c444=444, c445=445, c446=446, c447=447, c448=448, c449=449, c450=450, c451=451, c452=452, c453=453, c454=454, c455=455, c456=456, c457=457, c458=458, c459=459, c460=460, c461=461, c462=462, c463=463, c464=464, c465=465, c466=466, c467=467, c468=468, c469=469, c470=470, c471=471, c472=472, c473=473, c474=474, c475=475, c476=476, c477=477, c478=478, c479=479, c480=480, c481=481, c482=482, c483=483, c484=484, c485=485, c486=486, c487=487, c488=488, c489=489, c490=490, c491=491, c492=492, c493=493, c494=494, c495=495, c496=496, c497=497, c498=498, c499=499, c500=500, c501=501, c502=502, c503=503, c504=504, c505=505, c506=506, c507=507, c508=508, c509=509, c510=510, c511=511, c512=512, c513=513, c514=514, c515=515, c516=516, c517=517, c518=518, c519=519, c520=520, c521=521, c522=522, c523=523, c524=524, c525=525, c526=526, c527=527, c528=528, c529=529, c530=530, c531=531, c532=532, c533=533, c534=534, c535=535, c536=536, c537=537, c538=538, c539=539, c540=540, c541=541, c542=542, c543=543, c544=544, c545=545, c546=546, c547=547, c548=548, c549=549, c550=550, c551=551, c552=552, c553=553, c554=554, c555=555, c556=556, c557=557, c558=558, c559=559, c560=560, c561=561, c562=562, c563=563, c564=564, c565=565, c566=566, c567=567, c568=568, c569=569, c570=570, c571=571, c572=572, c573=573, c574=574, c575=575, c576=576, c577=577, c578=578, c579=579, c580=580, c581=581, c582=582, c583=583, c584=584, c585=585, c586=586, c587=587, c588=588, c589=589, c590=590, c591=591, c592=592, c593=593, c594=594, c595=595, c596=596, c597=597, c598=598, c599=599, c600=600, c601=601, c602=602, c603=603, c604=604, c605=605, c606=606, c607=607, c608=608, c609=609, c610=610, c611=611, c612=612, c613=613, c614=614, c615=615, c616=616, c617=617, c618=618, c619=619, c620=620, c621=621, c622=622, c623=623, c624=624, c625=625, c626=626, c627=627, c628=628, c629=629, c630=630, c631=631, c632=632, c633=633, c634=634, c635=635, c636=636, c637=637, c638=638, c639=639, c640=640, c641=641, c642=642, c643=643, c644=644, c645=645, c646=646, c647=647, c648=648, c649=649, c650=650, c651=651, c652=652, c653=653, c654=654, c655=655, c656=656, c657=657, c658=658, c659=659, c660=660, c661=661, c662=662, c663=663, c664=664, c665=665, c666=666, c667=667, c668=668, c669=669, c670=670, c671=671, c672=672, c673=673, c674=674, c675=675, c676=676, c677=677, c678=678, c679=679, c680=680, c681=681, c682=682, c683=683, c684=684, c685=685, c686=686, c687=687, c688=688, c689=689, c690=690, c691=691, c692=692, c693=693, c694=694, c695=695, c696=696, c697=697, c698=698, c699=699, c700=700, c701=701, c702=702, c703=703, c704=704, c705=705, c706=706, c707=707, c708=708, c709=709, c710=710, c711=711, c712=712, c713=713, c714=714, c715=715, c716=716, c717=717, c718=718, c719=719, c720=720, c721=721, c722=722, c723=723, c724=724, c725=725, c726=726, c727=727, c728=728, c729=729, c730=730, c731=731, c732=732, c733=733, c734=734, c735=735, c736=736, c737=737, c738=738, c739=739, c740=740, c741=741, c742=742, c743=743, c744=744, c745=745, c746=746, c747=747, c748=748, c749=749, c750=750, c751=751, c752=752, c753=753, c754=754, c755=755, c756=756, c757=757, c758=758, c759=759, c760=760, c761=761, c762=762, c763=763, c764=764, c765=765, c766=766, c767=767, c768=768, c769=769, c770=770, c771=771, c772=772, c773=773, c774=774, c775=775, c776=776, c777=777, c778=778, c779=779, c780=780, c781=781, c782=782, c783=783, c784=784, c785=785, c786=786, c787=787, c788=788, c789=789, c790=790, c791=791, c792=792, c793=793, c794=794, c795=795, c796=796, c797=797, c798=798, c799=799, c800=800, c801=801, c802=802, c803=803, c804=804, c805=805, c806=806, c807=807, c808=808, c809=809, c810=810, c811=811, c812=812, c813=813, c814=814, c815=815, c816=816, c817=817, c818=818, c819=819, c820=820, c821=821, c822=822, c823=823, c824=824, c825=825, c826=826, c827=827, c828=828, c829=829, c830=830, c831=831, c832=832, c833=833, c834=834, c835=835, c836=836, c837=837, c838=838, c839=839, c840=840, c841=841, c842=842, c843=843, c844=844, c845=845, c846=846, c847=847, c848=848, c849=849, c850=850, c851=851, c852=852, c853=853, c854=854, c855=855, c856=856, c857=857, c858=858, c859=859, c860=860, c861=861, c862=862, c863=863, c864=864, c865=865, c866=866, c867=867, c868=868, c869=869, c870=870, c871=871, c872=872, c873=873, c874=874, c875=875, c876=876, c877=877, c878=878, c879=879, c880=880, c881=881, c882=882, c883=883, c884=884, c885=885, c886=886, c887=887, c888=888, c889=889, c890=890, c891=891, c892=892, c893=893, c894=894, c895=895, c896=896, c897=897, c898=898, c899=899, c900=900, c901=901, c902=902, c903=903, c904=904, c905=905, c906=906, c907=907, c908=908, c909=909, c910=910, c911=911, c912=912, c913=913, c914=914, c915=915, c916=916, c917=917, c918=918, c919=919, c920=920, c921=921, c922=922, c923=923, c924=924, c925=925, c926=926, c927=927, c928=928, c929=929, c930=930, c931=931, c932=932, c933=933, c934=934, c935=935, c936=936, c937=937, c938=938, c939=939, c940=940, c941=941, c942=942, c943=943, c944=944, c945=945, c946=946, c947=947, c948=948, c949=949, c950=950, c951=951, c952=952, c953=953, c954=954, c955=955, c956=956, c957=957, c958=958, c959=959, c960=960, c961=961, c962=962, c963=963, c964=964, c965=965, c966=966, c967=967, c968=968, c969=969, c970=970, c971=971, c972=972, c973=973, c974=974, c975=975, c976=976, c977=977, c978=978, c979=979, c980=980, c981=981, c982=982, c983=983, c984=984, c985=985, c986=986, c987=987, c988=988, c989=989, c990=990, c991=991, c992=992, c993=993, c994=994, c995=995, c996=996, c997=997, c998=998, c999=999, out=100)
+Row(c0=0, c1=0, c2=0, c3=0, c4=0, c5=0, c6=0, c7=0, c8=0, c9=0, c10=0, c11=0, c12=0, c13=0, c14=0, c15=0, c16=0, c17=0, c18=0, c19=0, c20=0, c21=0, c22=0, c23=0, c24=0, c25=0, c26=0, c27=0, c28=0, c29=0, c30=0, c31=0, c32=0, c33=0, c34=0, c35=0, c36=0, c37=0, c38=0, c39=0, c40=0, c41=0, c42=0, c43=0, c44=0, c45=0, c46=0, c47=0, c48=0, c49=0, c50=0, c51=0, c52=0, c53=0, c54=0, c55=0, c56=0, c57=0, c58=0, c59=0, c60=0, c61=0, c62=0, c63=0, c64=0, c65=0, c66=0, c67=0, c68=0, c69=0, c70=0, c71=0, c72=0, c73=0, c74=0, c75=0, c76=0, c77=0, c78=0, c79=0, c80=0, c81=0, c82=0, c83=0, c84=0, c85=0, c86=0, c87=0, c88=0, c89=0, c90=0, c91=0, c92=0, c93=0, c94=0, c95=0, c96=0, c97=0, c98=0, c99=0, c100=0, c101=0, c102=0, c103=0, c104=0, c105=0, c106=0, c107=0, c108=0, c109=0, c110=0, c111=0, c112=0, c113=0, c114=0, c115=0, c116=0, c117=0, c118=0, c119=0, c120=0, c121=0, c122=0, c123=0, c124=0, c125=0, c126=0, c127=0, c128=0, c129=0, c130=0, c131=0, c132=0, c133=0, c134=0, c135=0, c136=0, c137=0, c138=0, c139=0, c140=0, c141=0, c142=0, c143=0, c144=0, c145=0, c146=0, c147=0, c148=0, c149=0, c150=0, c151=0, c152=0, c153=0, c154=0, c155=0, c156=0, c157=0, c158=0, c159=0, c160=0, c161=0, c162=0, c163=0, c164=0, c165=0, c166=0, c167=0, c168=0, c169=0, c170=0, c171=0, c172=0, c173=0, c174=0, c175=0, c176=0, c177=0, c178=0, c179=0, c180=0, c181=0, c182=0, c183=0, c184=0, c185=0, c186=0, c187=0, c188=0, c189=0, c190=0, c191=0, c192=0, c193=0, c194=0, c195=0, c196=0, c197=0, c198=0, c199=0, c200=0, c201=0, c202=0, c203=0, c204=0, c205=0, c206=0, c207=0, c208=0, c209=0, c210=0, c211=0, c212=0, c213=0, c214=0, c215=0, c216=0, c217=0, c218=0, c219=0, c220=0, c221=0, c222=0, c223=0, c224=0, c225=0, c226=0, c227=0, c228=0, c229=0, c230=0, c231=0, c232=0, c233=0, c234=0, c235=0, c236=0, c237=0, c238=0, c239=0, c240=0, c241=0, c242=0, c243=0, c244=0, c245=0, c246=0, c247=0, c248=0, c249=0, c250=0, c251=0, c252=0, c253=0, c254=0, c255=0, c256=0, c257=0, c258=0, c259=0, c260=0, c261=0, c262=0, c263=0, c264=0, c265=0, c266=0, c267=0, c268=0, c269=0, c270=0, c271=0, c272=0, c273=0, c274=0, c275=0, c276=0, c277=0, c278=0, c279=0, c280=0, c281=0, c282=0, c283=0, c284=0, c285=0, c286=0, c287=0, c288=0, c289=0, c290=0, c291=0, c292=0, c293=0, c294=0, c295=0, c296=0, c297=0, c298=0, c299=0, c300=0, c301=0, c302=0, c303=0, c304=0, c305=0, c306=0, c307=0, c308=0, c309=0, c310=0, c311=0, c312=0, c313=0, c314=0, c315=0, c316=0, c317=0, c318=0, c319=0, c320=0, c321=0, c322=0, c323=0, c324=0, c325=0, c326=0, c327=0, c328=0, c329=0, c330=0, c331=0, c332=0, c333=0, c334=0, c335=0, c336=0, c337=0, c338=0, c339=0, c340=0, c341=0, c342=0, c343=0, c344=0, c345=0, c346=0, c347=0, c348=0, c349=0, c350=0, c351=0, c352=0, c353=0, c354=0, c355=0, c356=0, c357=0, c358=0, c359=0, c360=0, c361=0, c362=0, c363=0, c364=0, c365=0, c366=0, c367=0, c368=0, c369=0, c370=0, c371=0, c372=0, c373=0, c374=0, c375=0, c376=0, c377=0, c378=0, c379=0, c380=0, c381=0, c382=0, c383=0, c384=0, c385=0, c386=0, c387=0, c388=0, c389=0, c390=0, c391=0, c392=0, c393=0, c394=0, c395=0, c396=0, c397=0, c398=0, c399=0, c400=0, c401=0, c402=0, c403=0, c404=0, c405=0, c406=0, c407=0, c408=0, c409=0, c410=0, c411=0, c412=0, c413=0, c414=0, c415=0, c416=0, c417=0, c418=0, c419=0, c420=0, c421=0, c422=0, c423=0, c424=0, c425=0, c426=0, c427=0, c428=0, c429=0, c430=0, c431=0, c432=0, c433=0, c434=0, c435=0, c436=0, c437=0, c438=0, c439=0, c440=0, c441=0, c442=0, c443=0, c444=0, c445=0, c446=0, c447=0, c448=0, c449=0, c450=0, c451=0, c452=0, c453=0, c454=0, c455=0, c456=0, c457=0, c458=0, c459=0, c460=0, c461=0, c462=0, c463=0, c464=0, c465=0, c466=0, c467=0, c468=0, c469=0, c470=0, c471=0, c472=0, c473=0, c474=0, c475=0, c476=0, c477=0, c478=0, c479=0, c480=0, c481=0, c482=0, c483=0, c484=0, c485=0, c486=0, c487=0, c488=0, c489=0, c490=0, c491=0, c492=0, c493=0, c494=0, c495=0, c496=0, c497=0, c498=0, c499=0, c500=0, c501=0, c502=0, c503=0, c504=0, c505=0, c506=0, c507=0, c508=0, c509=0, c510=0, c511=0, c512=0, c513=0, c514=0, c515=0, c516=0, c517=0, c518=0, c519=0, c520=0, c521=0, c522=0, c523=0, c524=0, c525=0, c526=0, c527=0, c528=0, c529=0, c530=0, c531=0, c532=0, c533=0, c534=0, c535=0, c536=0, c537=0, c538=0, c539=0, c540=0, c541=0, c542=0, c543=0, c544=0, c545=0, c546=0, c547=0, c548=0, c549=0, c550=0, c551=0, c552=0, c553=0, c554=0, c555=0, c556=0, c557=0, c558=0, c559=0, c560=0, c561=0, c562=0, c563=0, c564=0, c565=0, c566=0, c567=0, c568=0, c569=0, c570=0, c571=0, c572=0, c573=0, c574=0, c575=0, c576=0, c577=0, c578=0, c579=0, c580=0, c581=0, c582=0, c583=0, c584=0, c585=0, c586=0, c587=0, c588=0, c589=0, c590=0, c591=0, c592=0, c593=0, c594=0, c595=0, c596=0, c597=0, c598=0, c599=0, c600=0, c601=0, c602=0, c603=0, c604=0, c605=0, c606=0, c607=0, c608=0, c609=0, c610=0, c611=0, c612=0, c613=0, c614=0, c615=0, c616=0, c617=0, c618=0, c619=0, c620=0, c621=0, c622=0, c623=0, c624=0, c625=0, c626=0, c627=0, c628=0, c629=0, c630=0, c631=0, c632=0, c633=0, c634=0, c635=0, c636=0, c637=0, c638=0, c639=0, c640=0, c641=0, c642=0, c643=0, c644=0, c645=0, c646=0, c647=0, c648=0, c649=0, c650=0, c651=0, c652=0, c653=0, c654=0, c655=0, c656=0, c657=0, c658=0, c659=0, c660=0, c661=0, c662=0, c663=0, c664=0, c665=0, c666=0, c667=0, c668=0, c669=0, c670=0, c671=0, c672=0, c673=0, c674=0, c675=0, c676=0, c677=0, c678=0, c679=0, c680=0, c681=0, c682=0, c683=0, c684=0, c685=0, c686=0, c687=0, c688=0, c689=0, c690=0, c691=0, c692=0, c693=0, c694=0, c695=0, c696=0, c697=0, c698=0, c699=0, c700=0, c701=0, c702=0, c703=0, c704=0, c705=0, c706=0, c707=0, c708=0, c709=0, c710=0, c711=0, c712=0, c713=0, c714=0, c715=0, c716=0, c717=0, c718=0, c719=0, c720=0, c721=0, c722=0, c723=0, c724=0, c725=0, c726=0, c727=0, c728=0, c729=0, c730=0, c731=0, c732=0, c733=0, c734=0, c735=0, c736=0, c737=0, c738=0, c739=0, c740=0, c741=0, c742=0, c743=0, c744=0, c745=0, c746=0, c747=0, c748=0, c749=0, c750=0, c751=0, c752=0, c753=0, c754=0, c755=0, c756=0, c757=0, c758=0, c759=0, c760=0, c761=0, c762=0, c763=0, c764=0, c765=0, c766=0, c767=0, c768=0, c769=0, c770=0, c771=0, c772=0, c773=0, c774=0, c775=0, c776=0, c777=0, c778=0, c779=0, c780=0, c781=0, c782=0, c783=0, c784=0, c785=0, c786=0, c787=0, c788=0, c789=0, c790=0, c791=0, c792=0, c793=0, c794=0, c795=0, c796=0, c797=0, c798=0, c799=0, c800=0, c801=0, c802=0, c803=0, c804=0, c805=0, c806=0, c807=0, c808=0, c809=0, c810=0, c811=0, c812=0, c813=0, c814=0, c815=0, c816=0, c817=0, c818=0, c819=0, c820=0, c821=0, c822=0, c823=0, c824=0, c825=0, c826=0, c827=0, c828=0, c829=0, c830=0, c831=0, c832=0, c833=0, c834=0, c835=0, c836=0, c837=0, c838=0, c839=0, c840=0, c841=0, c842=0, c843=0, c844=0, c845=0, c846=0, c847=0, c848=0, c849=0, c850=0, c851=0, c852=0, c853=0, c854=0, c855=0, c856=0, c857=0, c858=0, c859=0, c860=0, c861=0, c862=0, c863=0, c864=0, c865=0, c866=0, c867=0, c868=0, c869=0, c870=0, c871=0, c872=0, c873=0, c874=0, c875=0, c876=0, c877=0, c878=0, c879=0, c880=0, c881=0, c882=0, c883=0, c884=0, c885=0, c886=0, c887=0, c888=0, c889=0, c890=0, c891=0, c892=0, c893=0, c894=0, c895=0, c896=0, c897=0, c898=0, c899=0, c900=0, c901=0, c902=0, c903=0, c904=0, c905=0, c906=0, c907=0, c908=0, c909=0, c910=0, c911=0, c912=0, c913=0, c914=0, c915=0, c916=0, c917=0, c918=0, c919=0, c920=0, c921=0, c922=0, c923=0, c924=0, c925=0, c926=0, c927=0, c928=0, c929=0, c930=0, c931=0, c932=0, c933=0, c934=0, c935=0, c936=0, c937=0, c938=0, c939=0, c940=0, c941=0, c942=0, c943=0, c944=0, c945=0, c946=0, c947=0, c948=0, c949=0, c950=0, c951=0, c952=0, c953=0, c954=0, c955=0, c956=0, c957=0, c958=0, c959=0, c960=0, c961=0, c962=0, c963=0, c964=0, c965=0, c966=0, c967=0, c968=0, c969=0, c970=0, c971=0, c972=0, c973=0, c974=0, c975=0, c976=0, c977=0, c978=0, c979=0, c980=0, c981=0, c982=0, c983=0, c984=0, c985=0, c986=0, c987=0, c988=0, c989=0, c990=0, c991=0, c992=0, c993=0, c994=0, c995=0, c996=0, c997=0, c998=0, c999=0, out=0)
...
FAILED ../../src/main/python/parquet_test.py::test_many_column_project[DATAGEN_SEED=1740536813, TZ=UTC, INJECT_OOM] - AssertionError: GPU and CPU int values are different at [0, 'c1']

Steps/Code to reproduce bug
Please provide a list of steps or a code sample to reproduce the issue.
Avoid posting private or sensitive data.

Expected behavior
A clear and concise description of what you expected to happen.

Environment details (please complete the following information)
GB100 + cuda12.8

Revision

  • spark-rapids: 1b0bce5 (branch-25.02)
  • spark-rapids-jni: f6fd3f96f286b88d3e6e7bdf2a13dede753b2d09 (branch-25.02)
  • cudf: 1fe744fb43044e9beb728c244f3f4a0beac3588f (branch-25.02)

Additional context
Add any other context about the problem here.

@yinqingh yinqingh added ? - Needs Triage Need team to review and classify bug Something isn't working labels Feb 26, 2025
@pxLi
Copy link
Member

pxLi commented Feb 26, 2025

cc @ttnghia @sameerz for visibility

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
? - Needs Triage Need team to review and classify bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants