Use bf16/fp16 data type for inductor vectorized data type

[CaoE]

cc @jgong5 @mingfeima @XiaobingSuper @sanchitintel @ashokei @jingxu10 @voznesenskym @penguinwu @EikanWang @Guobing-Chen @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @peterbell10 @ipiszy @yf225 @chenyang78 @kadeng @muchulee8 @ColinPeppler @amjames @desertfire @chauhang

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/126395

• 📄 Preview Python docs built from this PR
• 📄 Preview C++ docs built from this PR
• ❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

❌ 40 New Failures, 15 Unrelated Failures

As of commit 69a3181 with merge base ab07867 ():

NEW FAILURES - The following jobs have failed:

• inductor / rocm6.1-py3.8-inductor / test (inductor, 1, 1, linux.rocm.gpu.2) (gh)
  inductor/test_aot_inductor.py::AOTInductorTestABICompatibleCpu::test_zero_grid_with_unbacked_symbols_abi_compatible_cpu
• Lint / lintrunner-noclang / linux-job (gh)
  >>> Lint for torch/_inductor/codegen/cpp.py:
• periodic / linux-focal-cuda11.8-py3.10-gcc9-debug / test (default, 2, 5, linux.4xlarge.nvidia.gpu) (gh)
  inductor/test_torchinductor_dynamic_shapes.py::DynamicShapesCpuTests::test_bfloat16_to_int16_dynamic_shapes_cpu
• periodic / linux-focal-cuda11.8-py3.10-gcc9-debug / test (default, 3, 5, linux.4xlarge.nvidia.gpu) (gh)
  inductor/test_torchinductor.py::CpuTests::test_dist_bf16_cpu
• periodic / linux-focal-cuda11.8-py3.10-gcc9-debug / test (default, 4, 5, linux.4xlarge.nvidia.gpu) (gh)
  inductor/test_cpu_repro.py::CPUReproTests::test_atomic_add_lowp_fp
• periodic / linux-focal-cuda11.8-py3.10-gcc9-debug / test (default, 5, 5, linux.4xlarge.nvidia.gpu) (gh)
  inductor/test_cpu_repro.py::CPUReproTests::test_redundant_to_node_elimination_lowp_fp
• periodic / parallelnative-linux-jammy-py3.8-gcc11 / test (default, 1, 3, linux.2xlarge) (gh)
  inductor/test_torchinductor_dynamic_shapes.py::DynamicShapesCpuTests::test_adaptive_avg_pool2d_low_prec_dynamic_shapes_cpu
• periodic / parallelnative-linux-jammy-py3.8-gcc11 / test (default, 2, 3, linux.2xlarge) (gh)
  inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCPU::test_comprehensive_empty_cpu_float16
• periodic / parallelnative-linux-jammy-py3.8-gcc11 / test (default, 3, 3, linux.2xlarge) (gh)
  inductor/test_torchinductor.py::CpuTests::test_multilayer_var_lowp_cpu
• pull / linux-focal-cuda12.1-py3.10-gcc9 / test (default, 1, 5, linux.4xlarge.nvidia.gpu) (gh)
  inductor/test_cpu_repro.py::CPUReproTests::test_atomic_add_lowp_fp
• pull / linux-focal-cuda12.1-py3.10-gcc9 / test (default, 2, 5, linux.4xlarge.nvidia.gpu) (gh)
  inductor/test_torchinductor_dynamic_shapes.py::DynamicShapesCpuTests::test_adaptive_avg_pool2d_low_prec_dynamic_shapes_cpu
• pull / linux-focal-cuda12.1-py3.10-gcc9 / test (default, 3, 5, linux.4xlarge.nvidia.gpu) (gh)
  inductor/test_torchinductor_dynamic_shapes.py::DynamicShapesCpuTests::test_full_bits_lowp_dynamic_shapes_cpu
• pull / linux-focal-cuda12.1-py3.10-gcc9 / test (default, 4, 5, linux.4xlarge.nvidia.gpu) (gh)
  inductor/test_torchinductor_dynamic_shapes.py::DynamicShapesCpuTests::test_cat_upcasting_dynamic_shapes_cpu
• pull / linux-focal-cuda12.1-py3.10-gcc9 / test (default, 5, 5, linux.4xlarge.nvidia.gpu) (gh)
  inductor/test_torchinductor_dynamic_shapes.py::DynamicShapesCpuTests::test_bfloat16_to_int16_dynamic_shapes_cpu
• pull / linux-focal-py3.11-clang10 / test (crossref, 1, 2, linux.2xlarge) (gh)
  inductor/test_mmdecomp.py::TestDecompCPU::test_simple_mm_bfloat16_cpu_bfloat16
• pull / linux-focal-py3.11-clang10 / test (default, 1, 3, linux.2xlarge) (gh)
  inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCPU::test_comprehensive___rsub___cpu_float16
• pull / linux-focal-py3.11-clang10 / test (default, 2, 3, linux.2xlarge) (gh)
  inductor/test_cpu_repro.py::CPUReproTests::test_lstm_packed_change_input_sizes_cpu
• pull / linux-focal-py3.11-clang10 / test (default, 3, 3, linux.2xlarge) (gh)
  inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCPU::test_comprehensive___radd___cpu_float16
• pull / linux-focal-py3.12-clang10 / test (default, 1, 3, linux.2xlarge) (gh)
  inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCPU::test_comprehensive_atan2_cpu_float16
• pull / linux-focal-py3.12-clang10 / test (default, 2, 3, linux.2xlarge) (gh)
  inductor/test_cpu_repro.py::CPUReproTests::test_lstm_packed_change_input_sizes_cpu
• pull / linux-focal-py3.12-clang10 / test (default, 3, 3, linux.2xlarge) (gh)
  inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCPU::test_comprehensive___rmatmul___cpu_float16
• pull / linux-focal-py3.8-clang10 / test (crossref, 1, 2, linux.2xlarge) (gh)
  inductor/test_mmdecomp.py::TestDecompCPU::test_simple_mm_bfloat16_cpu_bfloat16
• pull / linux-focal-py3.8-clang10 / test (default, 1, 3, linux.2xlarge) (gh)
  inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCPU::test_comprehensive___radd___cpu_float16
• pull / linux-focal-py3.8-clang10 / test (default, 2, 3, linux.2xlarge) (gh)
  inductor/test_cpu_repro.py::CPUReproTests::test_lstm_packed_change_input_sizes_cpu
• pull / linux-focal-py3.8-clang10 / test (default, 3, 3, linux.2xlarge) (gh)
  inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCPU::test_comprehensive___rmul___cpu_float16
• pull / linux-jammy-py3.10-clang15-asan / test (default, 1, 6, linux.4xlarge) (gh)
  inductor/test_cpu_repro.py::CPUReproTests::test_lstm_packed_change_input_sizes_cpu
• pull / linux-jammy-py3.10-clang15-asan / test (default, 2, 6, linux.4xlarge) (gh)
  inductor/test_cpu_repro.py::CPUReproTests::test_atomic_add_lowp_fp
• pull / linux-jammy-py3.10-clang15-asan / test (default, 3, 6, linux.4xlarge) (gh)
  inductor/test_torchinductor_codegen_dynamic_shapes.py::DynamicShapesCodegenCpuTests::test_cat_upcasting_dynamic_shapes_cpu
• pull / linux-jammy-py3.10-clang15-asan / test (default, 4, 6, linux.4xlarge) (gh)
  inductor/test_torchinductor_codegen_dynamic_shapes.py::DynamicShapesCodegenCpuTests::test_add_complex4_dynamic_shapes_cpu
• pull / linux-jammy-py3.10-clang15-asan / test (default, 5, 6, linux.4xlarge) (gh)
  inductor/test_torchinductor_codegen_dynamic_shapes.py::DynamicShapesCodegenCpuTests::test_bfloat16_to_int16_dynamic_shapes_cpu
• pull / linux-jammy-py3.10-clang15-asan / test (default, 6, 6, linux.4xlarge) (gh)
  inductor/test_torchinductor_codegen_dynamic_shapes.py::DynamicShapesCodegenCpuTests::test_adaptive_avg_pool2d_low_prec_dynamic_shapes_cpu
• pull / linux-jammy-py3.8-gcc11 / test (default, 1, 3, linux.2xlarge) (gh)
  inductor/test_cpu_repro.py::CPUReproTests::test_atomic_add_lowp_fp
• pull / linux-jammy-py3.8-gcc11 / test (default, 2, 3, linux.2xlarge) (gh)
  inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCPU::test_comprehensive___radd___cpu_float16
• pull / linux-jammy-py3.8-gcc11 / test (default, 3, 3, linux.2xlarge) (gh)
  inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCPU::test_comprehensive___rsub___cpu_float16
• trunk / linux-focal-cuda12.1-py3.10-gcc9 / test (nogpu_AVX512, 1, 1, linux.2xlarge) (gh)
  inductor/test_torchinductor.py::CpuTests::test_dist_bf16_cpu
• trunk / linux-focal-cuda12.1-py3.10-gcc9 / test (nogpu_NO_AVX2, 1, 1, linux.2xlarge) (gh)
  inductor/test_torchinductor.py::CpuTests::test_dist_bf16_cpu
• trunk / linux-focal-rocm6.1-py3.8 / test (default, 1, 2, linux.rocm.gpu) (gh)
  inductor/test_torchinductor.py::CpuTests::test_dist_bf16_cpu
• trunk / linux-focal-rocm6.1-py3.8 / test (default, 2, 2, linux.rocm.gpu) (gh)
  inductor/test_torchinductor.py::GPUTests::test_full_bits_lowp_cuda
• trunk / macos-13-py3-arm64 / test (default, 2, 3, macos-m1-stable) (gh)
  inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCPU::test_comprehensive___radd___cpu_float16
• trunk / macos-13-py3-arm64 / test (default, 3, 3, macos-m1-stable) (gh)
  inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCPU::test_comprehensive___rmul___cpu_float16

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

• inductor / linux-jammy-cpu-py3.8-gcc11-inductor / test (inductor_torchbench_cpu_smoketest_perf, 1, 1, linux.24xl.spr-metal) (gh) (similar failure)
  Process completed with exit code 1.
• periodic / win-vs2019-cuda11.8-py3 / test (default, 4, 4, windows.g5.4xlarge.nvidia.gpu) (gh) (matched win rule in flaky-rules.json)
  The process cannot access the file 'C:\actions-runner\_work\_actions\pytorch\pytorch\main\aten\src\ATen\core\dispatch' because it is being used by another process.

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• inductor-periodic / cuda12.1-py3.10-gcc9-sm86-periodic-dynamo-benchmarks / test (aot_eager_torchbench, 1, 2, linux.g5.4xlarge.nvidia.gpu) (gh) (trunk failure)
  hf_BigBird
• inductor-periodic / cuda12.1-py3.10-gcc9-sm86-periodic-dynamo-benchmarks / test (dynamic_aot_eager_torchbench, 1, 2, linux.g5.4xlarge.nvidia.gpu) (gh) (trunk failure)
  hf_BigBird
• inductor-periodic / cuda12.1-py3.10-gcc9-sm86-periodic-dynamo-benchmarks / test (dynamo_eager_torchbench, 1, 2, linux.g5.4xlarge.nvidia.gpu) (gh) (trunk failure)
  hf_BigBird
• periodic / linux-focal-cuda11.8-py3.9-gcc9 / test (multigpu, 1, 1, linux.g5.12xlarge.nvidia.gpu) (gh) (trunk failure)
  !!!!!!!!!!!!!!!!!!!!!!!!!!!!!! KeyboardInterrupt !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
• periodic / linux-focal-rocm6.1-py3.8 / test (distributed, 1, 2, linux.rocm.gpu) (gh) (trunk failure)
  distributed/_tensor/test_attention.py::RingAttentionTest::test_ring_attention_sdpa_is_causal_False

UNSTABLE - The following jobs failed but were likely due to flakiness present on trunk and has been marked as unstable:

• inductor / cuda12.1-py3.10-gcc9-sm86 / test (aot_inductor_torchbench, 1, 2, linux.g5.4xlarge.nvidia.gpu, unstable) (gh) (#120841)
  hf_BigBird
• inductor / cuda12.1-py3.10-gcc9-sm86 / test (dynamic_inductor_torchbench, 1, 2, linux.g5.4xlarge.nvidia.gpu, unstable) (gh) (#120841)
  hf_BigBird
• inductor / cuda12.1-py3.10-gcc9-sm86 / test (inductor_torchbench, 1, 2, linux.g5.4xlarge.nvidia.gpu, unstable) (gh) (#120841)
  hf_BigBird
• inductor / cuda12.1-py3.10-gcc9-sm86 / test (inductor, 1, 1, linux.g5.4xlarge.nvidia.gpu, unstable) (gh) (#120841)
  inductor/test_aot_inductor.py::AOTInductorTestABICompatibleCpuWithStackAllocation::test_zero_grid_with_unbacked_symbols_abi_compatible_cpu_with_stack_allocation
• pull / linux-focal-cuda12.1-py3.10-gcc9-sm86 / test (default, 2, 5, linux.g5.4xlarge.nvidia.gpu, unstable) (gh) ()
  inductor/test_cpu_repro.py::CPUReproTests::test_atomic_add_lowp_fp
• pull / linux-focal-cuda12.1-py3.10-gcc9-sm86 / test (default, 3, 5, linux.g5.4xlarge.nvidia.gpu, unstable) (gh) ()
  inductor/test_torchinductor_codegen_dynamic_shapes.py::DynamicShapesCodegenCpuTests::test_adaptive_avg_pool2d_low_prec_dynamic_shapes_cpu
• pull / linux-focal-cuda12.1-py3.10-gcc9-sm86 / test (default, 4, 5, linux.g5.4xlarge.nvidia.gpu, unstable) (gh) ()
  inductor/test_torchinductor_dynamic_shapes.py::DynamicShapesCpuTests::test_cat_upcasting_dynamic_shapes_cpu
• pull / linux-focal-cuda12.1-py3.10-gcc9-sm86 / test (default, 5, 5, linux.g5.4xlarge.nvidia.gpu, unstable) (gh) ()
  inductor/test_torchinductor_dynamic_shapes.py::DynamicShapesCpuTests::test_add_complex4_dynamic_shapes_cpu

This comment was automatically generated by Dr. CI and updates every 15 minutes.