Merge upstream#77

Open

fsx950223 wants to merge 665 commits intoamd-developfrom merge_upstream

+152,356-20,709

This pull request is big! We're only showing the most recent 250 commits

Commits on Apr 10, 2023

Fix memory pool using in GEMM profiler (facebookincubator#556 )

aakhundov
authored and
facebook-github-bot
committed

Commits on Apr 11, 2023

dynamic seq (facebookincubator#560 )

Terry Chen
authored and
facebook-github-bot
committed
Fix MSVC compiler complaints (facebookincubator#551 )

Alexandr Guzhva
authored and
facebook-github-bot
committed
Fix MSVC compiler narrowing conversion errors for cuda/gemm_epilogue_vistor (facebookincubator#552 )

Alexandr Guzhva
authored and
facebook-github-bot
committed
Add dynamic_seq_len and dynamic_num_head support in b2b bmm. (facebookincubator#530 )

ipiszy
authored and
facebook-github-bot
committed

Commits on Apr 12, 2023

Commits on Apr 13, 2023

add conv1d op (facebookincubator#562 )

tenpercent
authored and
facebook-github-bot
committed
arange as model param (facebookincubator#566 )

mortzur
authored and
facebook-github-bot
committed
fixed an infinite loop in move_view_ops transformation (facebookincubator#570 )

chenyang78
authored and
facebook-github-bot
committed
Remove hack in [fx/aten]2ait (facebookincubator#516 )

muchulee8
authored and
facebook-github-bot
committed
Fix _fuse_strided_op_and_cat: no GEMM+concat fusion with dim>=rank (facebookincubator#559 )

Grigory Sizov
authored and
facebook-github-bot
committed

Commits on Apr 14, 2023

refactor
fsx950223
committed
merge updates
fsx950223
committed
Accept JaggedIntVar in padded_dense_to_jagged (facebookincubator#577 )

aakhundov
authored and
facebook-github-bot
committed
implement per op benchmark backbone (facebookincubator#555 )

tenpercent
authored and
facebook-github-bot
committed
Add SM90 kernel foundations (facebookincubator#575 )

aakhundov
authored and
facebook-github-bot
committed
add reshape for to converter (facebookincubator#576 )

frank-wei
authored and
facebook-github-bot
committed
Allow split without dim as input (facebookincubator#578 )

ZhengkaiZ
authored and
facebook-github-bot
committed

Commits on Apr 15, 2023

Replace infer_shape for split op. (facebookincubator#568 )

muchulee8
authored and
facebook-github-bot
committed
Add Identity op. (facebookincubator#579 )

muchulee8
authored and
facebook-github-bot
committed

Commits on Apr 17, 2023

Build cache: Do not cache failed builds (facebookincubator#586 )

kadeng
authored and
facebook-github-bot
committed
Add BatchNorm FE module (facebookincubator#587 )

Colin Chan
authored and
facebook-github-bot
committed
add identity for to and contiguous (facebookincubator#588 )

frank-wei
authored and
facebook-github-bot
committed

Commits on Apr 18, 2023

In-place weight updates tracing (facebookincubator#569 )

khabinov
authored and
facebook-github-bot
committed
Remove hardcoded dtype float16 in acc_ops_clone (facebookincubator#585 )

feikou
authored and
facebook-github-bot
committed

Commits on Apr 19, 2023

Fix acc_ops converter on std when keepdim=False (facebookincubator#593 )

tissue3
authored and
facebook-github-bot
committed

Commits on Apr 20, 2023

Fix race condition in FBCUDA (facebookincubator#591 )

kadeng
authored and
facebook-github-bot
committed
Upgrade CUTLASS to 3.1 (facebookincubator#584 )

aakhundov
authored and
facebook-github-bot
committed
benchmark: concat (facebookincubator#592 )

chenyang78
authored and
facebook-github-bot
committed

Commits on Apr 21, 2023

Ldm update (facebookincubator#573 )

terrychenism
authored and
facebook-github-bot
committed
Eliminated redundant permute pairs (facebookincubator#529 )

PivovarA
authored and
facebook-github-bot
committed

Commits on Apr 22, 2023

Back out "Ldm update" (facebookincubator#603 )

muchulee8
authored and
facebook-github-bot
committed
Add name for directory created by test_vanilla_attention (facebookincubator#602 )

muchulee8
authored and
facebook-github-bot
committed

Commits on Apr 23, 2023

Fix MSVC compiler narrowing conversion errors for cuda/gemm_epilogue_universal (facebookincubator#604 )

Alexandr Guzhva
authored and
facebook-github-bot
committed
Mask select converter (facebookincubator#601 )

wushirong
authored and
facebook-github-bot
committed

Commits on Apr 24, 2023

Skip incompatible tests outside CI (facebookincubator#606 )

aakhundov
authored and
facebook-github-bot
committed
Add env_variables context manager to test_utils (facebookincubator#607 )

aakhundov
authored and
facebook-github-bot
committed
Add CUDA_SM90 test env to test_utils (facebookincubator#608 )

aakhundov
authored and
facebook-github-bot
committed
Ldm update (facebookincubator#611 )

terrychenism
authored and
facebook-github-bot
committed
Add testname for test_var:test_batched_var (facebookincubator#612 )

muchulee8
authored and
facebook-github-bot
committed
Prepare for CUTLASS 3.x integration (facebookincubator#614 )

aakhundov
authored and
facebook-github-bot
committed

Commits on Apr 25, 2023

Set skip_on_empty=True from filter_test_cases_by_params (facebookincubator#609 )

aakhundov
authored and
facebook-github-bot
committed

Commits on Apr 26, 2023

Add MaxPool3d FE module (facebookincubator#595 )

Colin Chan
authored and
facebook-github-bot
committed
Add GELU FE module (facebookincubator#600 )

Colin Chan
authored and
facebook-github-bot
committed
Fill MultiScaleBlock gap (facebookincubator#622 )

Colin Chan
authored and
facebook-github-bot
committed
Update infer_shape for argmax (facebookincubator#623 )

muchulee8
authored and
facebook-github-bot
committed
move use_cuda from global to class CrossAttention level (facebookincubator#626 )

hl475
authored and
facebook-github-bot
committed
Introduce AIT_USE_FAST_MATH environment flag (facebookincubator#627 )

Alexandr Guzhva
authored and
facebook-github-bot
committed

Commits on Apr 27, 2023

Apply transform_permute_to_reshape when input tensor has smaller than required rank (facebookincubator#619 )

Grigory Sizov
authored and
facebook-github-bot
committed
use current stream in mem_eff_attention kernel (facebookincubator#628 )

mortzur
authored and
facebook-github-bot
committed
Minor fix to makefile normalization (facebookincubator#630 )

kadeng
authored and
facebook-github-bot
committed
Add SM90 CUTLASS 3.x kernels to gemm_rcr (facebookincubator#617 )

aakhundov
authored and
facebook-github-bot
committed
Add SM90 CUTLASS 3.x kernels to gemm_rrr (facebookincubator#621 )

aakhundov
authored and
facebook-github-bot
committed
Add SM90 CUTLASS 3.x kernels to gemm_rcr_bias (facebookincubator#620 )

aakhundov
authored and
facebook-github-bot
committed
Small fixes to b2b bmm kernels (facebookincubator#564 )

ipiszy
authored and
facebook-github-bot
committed

Commits on Apr 28, 2023

Fix AIT topk converter (facebookincubator#631 )

wushirong
authored and
facebook-github-bot
committed
Support fp32 accumulation (facebookincubator#632 )

kadeng
authored and
facebook-github-bot
committed
bias support (facebookincubator#633 )

frank-wei
authored and
facebook-github-bot
committed
Add PatchEmbed FE (facebookincubator#629 )

Colin Chan
authored and
facebook-github-bot
committed
Refactor test_bmm(_add) with filter_test_cases_by_test_env (facebookincubator#635 )

aakhundov
authored and
facebook-github-bot
committed

Commits on Apr 29, 2023

Update infer_shape for batch_gather (facebookincubator#624 )

muchulee8
authored and
facebook-github-bot
committed

Commits on Apr 30, 2023

Add SM90 CUTLASS 3.x kernels to bmm_xxx and bmm_xxx_add (facebookincubator#637 )

aakhundov
authored and
facebook-github-bot
committed
dynamic_slice infer_shape (facebookincubator#598 )

muchulee8
authored and
facebook-github-bot
committed

Commits on May 1, 2023

Replace hasattr with getattr in aitemplate/AITemplate/python/aitemplate/frontend/nn/multiscale_attention.py (facebookincubator#641 )

r-barnes
authored and
facebook-github-bot
committed

Commits on May 2, 2023

Fix circleci test errors (facebookincubator#639 )

ipiszy
authored and
facebook-github-bot
committed
Fix internal CI issue (facebookincubator#640 )

ipiszy
authored and
facebook-github-bot
committed

Commits on May 3, 2023

optionally print python callstack stats for model compilation (facebookincubator#642 )

max
authored and
facebook-github-bot
committed
Add VisionTransformerBasicHead (facebookincubator#636 )

Colin Chan
authored and
facebook-github-bot
committed
Duplicate LDM from Example to FB Folder (facebookincubator#646 )

henryhu6
authored and
facebook-github-bot
committed
Remove manually set Parameter names (facebookincubator#638 )

Colin Chan
authored and
facebook-github-bot
committed

Commits on May 4, 2023

classic_b2b_bmm: add multi-head and strides (facebookincubator#625 )

kadeng
authored and
facebook-github-bot
committed
Fix naming error caused by dump_program(). (facebookincubator#649 )

ipiszy
authored and
facebook-github-bot
committed
SD example fix (facebookincubator#654 )

terrychenism
authored and
facebook-github-bot
committed
remove eliminate_permutations for now (facebookincubator#655 )

chenyang78
authored and
facebook-github-bot
committed
jagged SHA and MHA module support (facebookincubator#651 )

frank-wei
authored and
facebook-github-bot
committed
Add BF16 support for ads model (facebookincubator#656 )

wushirong
authored and
facebook-github-bot
committed
_dlclose support for Windows (facebookincubator#657 )

Alexandr Guzhva
authored and
facebook-github-bot
committed

Commits on May 5, 2023

Add MultiscaleVisionTransformers FE + validate MVIT 21 block config E2E (facebookincubator#643 )

Colin Chan
authored and
facebook-github-bot
committed
Make GEMM ProfilerMemoryPool size computation generic (facebookincubator#653 )

aakhundov
authored and
facebook-github-bot
committed

Commits on May 6, 2023

Add Windows support for owned constants (facebookincubator#658 )

Alexandr Guzhva
authored and
facebook-github-bot
committed
simple multi-stream (facebookincubator#615 )

Alexandr Guzhva
authored and
facebook-github-bot
committed
Use device 0 for profiling if devices list is empty (facebookincubator#667 )

aakhundov
authored and
facebook-github-bot
committed

Commits on May 7, 2023

Allow setting jagged tensor total_length upper bound from offsets (facebookincubator#634 )

aakhundov
authored and
facebook-github-bot
committed
Increase CircleCI no_output_timeout to 20m (facebookincubator#668 )

aakhundov
authored and
facebook-github-bot
committed
Integrate CUDA 12.1 (facebookincubator#661 )

aakhundov
authored and
facebook-github-bot
committed
Sync CUTLASS with upstream (facebookincubator#662 )

aakhundov
authored and
facebook-github-bot
committed
Add SM90 CUTLASS 3.x kernels to gemm_rcr_bias_relu (facebookincubator#663 )

aakhundov
authored and
facebook-github-bot
committed

Commits on May 9, 2023

Fix test logging errors (facebookincubator#664 )

Colin Chan
authored and
facebook-github-bot
committed
fix bugs
fsx950223
committed
revert some changes
fsx950223
committed
format fx2ait code
fsx950223
committed
Add SM90 CUTLASS 3.x kernels to remaining gemm_rcr_bias_activation ops (facebookincubator#669 )

aakhundov
authored and
facebook-github-bot
committed
Add bf16 support to classic_b2b_bmm op (facebookincubator#665 )

aakhundov
authored and
facebook-github-bot
committed
merge_updates
fsx950223
committed
Add bf16 support to fmha_style_b2b_bmm op (facebookincubator#666 )

aakhundov
authored and
facebook-github-bot
committed
Windows test_standalone.py fix (facebookincubator#676 )

Alexandr Guzhva
authored and
facebook-github-bot
committed
Refactor infrastructure (facebookincubator#671 )

Alexandr Guzhva
authored and
facebook-github-bot
committed
introduce get_include_directories() in target_def.py files (facebookincubator#672 )

Alexandr Guzhva
authored and
facebook-github-bot
committed
Add is_view_of for identity op (facebookincubator#677 )

muchulee8
authored and
facebook-github-bot
committed

Commits on May 10, 2023

MSVC fix (facebookincubator#675 )

Alexandr Guzhva
authored and
facebook-github-bot
committed
Fix get_positive_dim for IntVar (facebookincubator#680 )

tissue3
authored and
facebook-github-bot
committed
Add bf16 support to full op (facebookincubator#679 )

tissue3
authored and
facebook-github-bot
committed

Commits on May 11, 2023

introduce get_host_compiler_options() and get_device_compiler_options() for Target class (facebookincubator#673 )

Alexandr Guzhva
authored and
facebook-github-bot
committed
speed up MultiScaleBlock (facebookincubator#681 )

Colin Chan
authored and
facebook-github-bot
committed
Introduce a CMake compiler engine (facebookincubator#674 )

Alexandr Guzhva
authored and
facebook-github-bot
committed
update ci (facebookincubator#589 )

fsx950223
authored and
facebook-github-bot
committed
Fix codegen condition check issue

wushirong
authored and
facebook-github-bot
committed

Commits on May 12, 2023

Commits on May 14, 2023

Fix a tricky fused_elementwise alignment issue. (facebookincubator#693 )

ipiszy
authored and
facebook-github-bot
committed

Commits on May 15, 2023

add ops
fsx950223
committed
Fix-forward - multiscaleblock test failure (facebookincubator#694 )

Colin Chan
authored and
facebook-github-bot
committed
Fix codegen duplicate dim decl issue (facebookincubator#692 )

qxy11
authored and
facebook-github-bot
committed
Add fast_math to LowerSettings and AIT Target (facebookincubator#695 )

ipiszy
authored and
facebook-github-bot
committed

Commits on May 16, 2023

fix bugs
fsx950223
committed
Fix use_fast_math (facebookincubator#697 )

ipiszy
authored and
facebook-github-bot
committed
Add layernorm CUDA kernel based on Welford's algorithm (facebookincubator#698 )

aakhundov
authored and
facebook-github-bot
committed

Commits on May 17, 2023

fix profiler group bug
fsx950223
committed
fix bugs
fsx950223
committed
CUDA debug log utility class (facebookincubator#688 )

kadeng
authored and
facebook-github-bot
committed

Commits on May 18, 2023

refactor sm detection; add quadro card name (facebookincubator#690 )

tenpercent
authored and
facebook-github-bot
committed
add nix-shell config (facebookincubator#691 )

tenpercent
authored and
facebook-github-bot
committed
Stable Diffusion dynamic input shape, include/exclude constants, load from diffusers/compvis, alternative pipeline (facebookincubator#696 )

hlky
authored and
facebook-github-bot
committed
Dump function properties at profiling time. (facebookincubator#684 )

ipiszy
authored and
facebook-github-bot
committed
add graph mode
fsx950223
committed
set target_has_graph_mode to true
fsx950223
committed
Add missing __init__.py files (facebookincubator#702 )

ymwangg
authored and
facebook-github-bot
committed

Commits on May 19, 2023

Commits on May 20, 2023

Add support for keyword argument in AITModule (facebookincubator#707 )

henryhu6
authored and
facebook-github-bot
committed

Commits on May 21, 2023

adjust launch config for group_layernorm kernels (facebookincubator#708 )

chenyang78
authored and
facebook-github-bot
committed
remove identity ops from the graph (facebookincubator#703 )

chenyang78
authored and
facebook-github-bot
committed

Commits on May 22, 2023

fx2ait: explicitly ensure workdir's existence (facebookincubator#714 )

kflu
authored and
facebook-github-bot
committed
enable bmm_rrr for concat + concat fusion (facebookincubator#716 )

chenyang78
authored and
facebook-github-bot
committed

Commits on May 23, 2023

support dynamic batch size
fsx950223
committed
fix a bug
fsx950223
committed
Sync CUTLASS version with upstream (facebookincubator#706 )

aakhundov
authored and
facebook-github-bot
committed
Merge branch 'merge_upstream_update' into merge_upstream
fsx950223
committed
Merge remote-tracking branch 'upstream/main' into mlperf
fsx950223
committed
check if the tensor is none before copying (facebookincubator#704 )

chengscott
authored and
facebook-github-bot
committed
Fix 02_vision_model example (facebookincubator#718 )

ymwangg
authored and
facebook-github-bot
committed
fix bert example (facebookincubator#710 )

tenpercent
authored and
facebook-github-bot
committed

Commits on May 24, 2023

Commits on May 25, 2023

nvcc options (facebookincubator#721 )

chengscott
authored and
facebook-github-bot
committed
2/n support layernorm and elementwise op (facebookincubator#711 )

frank-wei
authored and
facebook-github-bot
committed
fix bugs
fsx950223
committed
Merge remote-tracking branch 'upstream/main' into mlperf
fsx950223
committed
Back out "fix bert example" (facebookincubator#728 )

aakhundov
authored and
facebook-github-bot
committed
Stable Diffusion Alt fixes facebookincubator#722 facebookincubator#723 (facebookincubator#724 )

hlky
authored and
facebook-github-bot
committed

Commits on May 27, 2023

Sync CUTLASS with upstream again (facebookincubator#727 )

aakhundov
authored and
facebook-github-bot
committed

Commits on May 29, 2023

add missing script
fsx950223
committed
Split test_strided_layernorm test cases (facebookincubator#729 )

aakhundov
authored and
facebook-github-bot
committed

Commits on May 31, 2023

Commits on Jun 1, 2023

Commits on Jun 2, 2023

Grouped Classic B2B BMM op 2 ( padded -> jagged operator implementation ) (facebookincubator#737 )

kadeng
authored and
facebook-github-bot
committed
Add support of nn.functional.hardtanh (facebookincubator#739 )

mcremon-meta
authored and
facebook-github-bot
committed

Commits on Jun 6, 2023

Eiminate elementwise no-ops (*/1, +-0) (facebookincubator#746 )

AlbertDachiChen
authored and
facebook-github-bot
committed

Commits on Jun 7, 2023

Update README.md (facebookincubator#745 )

eltociear
authored and
facebook-github-bot
committed
Add PG509 as a detected CUDA GPU (facebookincubator#749 )

erjiang
authored and
facebook-github-bot
committed
Move XRayVideo related FE modules to frontend/nn + disambiguate test names (facebookincubator#748 )

Colin Chan
authored and
facebook-github-bot
committed

Commits on Jun 9, 2023

Commits on Jun 10, 2023

Fix SD compilation example to use user provided H and W (facebookincubator#755 )

apivovarov
authored and
facebook-github-bot
committed

Commits on Jun 11, 2023

Add --model-name param to SD download_pipeline script (facebookincubator#757 )

apivovarov
authored and
facebook-github-bot
committed
Include split+cat in fuse_split optimization (facebookincubator#740 )

erjiang
authored and
facebook-github-bot
committed

Commits on Jun 12, 2023

frontend.nn.attention dtype (facebookincubator#759 )

hlky
authored and
facebook-github-bot
committed
Increase recursion limit in dump_program using try except (facebookincubator#761 )

hl475
authored and
facebook-github-bot
committed
Daily arc lint --take BLACK

generatedunixname89002005367269
authored and
facebook-github-bot
committed
Map VAE params without AIT_AutoencoderKL (facebookincubator#760 )

hlky
authored and
facebook-github-bot
committed

Commits on Jun 13, 2023

Commits on Jun 14, 2023

Add ait profile timeout to lower_settings (facebookincubator#764 )

qxy11
authored and
facebook-github-bot
committed

Commits on Jun 15, 2023

Commits on Jun 16, 2023

Commits on Jun 19, 2023

Commits on Jun 20, 2023

rename to warp size
fsx950223
committed
native CUDA development helper (facebookincubator#772 )

kadeng
authored and
facebook-github-bot
committed
Standalone test cases (facebookincubator#773 )

kadeng
authored and
facebook-github-bot
committed

Commits on Jun 21, 2023

pool2d upstream (facebookincubator#775 )

fsx950223
authored and
facebook-github-bot
committed
upstream norm (facebookincubator#774 )

fsx950223
authored and
facebook-github-bot
committed

Commits on Jun 23, 2023

Download SD model without token by default (facebookincubator#769 )

apivovarov
authored and
facebook-github-bot
committed

Commits on Jun 24, 2023

Commits on Jun 25, 2023

tensor upstream (facebookincubator#776 )

fsx950223
authored and
facebook-github-bot
committed

Commits on Jun 26, 2023

Support List[Tensor] arg for from_two_input_lists (facebookincubator#792 )

henryhu6
authored and
facebook-github-bot
committed

Commits on Jun 27, 2023

Adding support for relational operations (facebookincubator#783 )

AlbertDachiChen
authored and
facebook-github-bot
committed
skip fusing slice with a strided op if the relevant tensor accessors are already set (facebookincubator#799 )

chenyang78
authored and
facebook-github-bot
committed
Add support for where operator (facebookincubator#791 )

AlbertDachiChen
authored and
facebook-github-bot
committed
Add test cases to cover split_large_slice_scatter (facebookincubator#794 )

y-sq
authored and
facebook-github-bot
committed
Add init_random_weights test util (facebookincubator#800 )

Colin Chan
authored and
facebook-github-bot
committed
Sync CUTLASS with upstream (facebookincubator#803 )

aakhundov
authored and
facebook-github-bot
committed

Commits on Jun 29, 2023

Fix SD Alternative pipeline README to use demo_alt.py (facebookincubator#784 )

apivovarov
authored and
facebook-github-bot
committed

Commits on Jun 30, 2023

Split slice_scatter into multiple ones if it has too many inputs (facebookincubator#801 )

y-sq
authored and
facebook-github-bot
committed
update frontend and mk_ck_lib (facebookincubator#777 )

fsx950223
authored and
facebook-github-bot
committed
merge upstream
fsx950223
committed

Commits on Oct 19, 2023

update dockerfile
fsx950223
committed

Commits on Oct 24, 2023

fix a profiler bug
fsx950223
committed

Commits on Nov 24, 2023

fix dockerfile
fsx950223
committed

Commits on Dec 12, 2023

pin diffusers and transformers
dejay-vu
committed

Commits on Dec 22, 2023

merge sdxl
fsx950223
committed

Commits on Dec 25, 2023

fix bugs
fsx950223
committed

Commits on Jan 3, 2024

Commits on Jan 4, 2024

Commits on Jan 5, 2024

Commits on Jan 9, 2024

fix profile bug
fsx950223
committed

Commits on Jan 10, 2024

fix profiler
fsx950223
committed

Commits on Jan 17, 2024

add interwave and pipeline tuning
fsx950223
committed

Commits on Jan 18, 2024

fix a profiler bug
fsx950223
committed

Commits on Jan 30, 2024

fix revision
fsx950223
committed