Ei`SSKrSSKJr SSKrSSKJr SSKJrJr SSK J r J r SSK J r SSKJrJr SS KJr SS KJrJrJr SS KJrJr SS KJr \R6"\5r\"\R<S SS9r\"\R@RBRDSS\RFS9r$\R@RBr!\R@RJr%\R@RLr&SSjr'SSjr(g)N)Any)mm_args)configlowering)CppGemmTemplateCppWoqInt4GemmTemplate)create_epilogue_with_attr)expandregister_lowering)WeightInt4PackMatmul)autotune_select_algorithmExternKernelChoicerealize_inputs)use_aten_gemm_kernelsuse_cpp_gemm_template)Vzat::_weight_int8pack_mmF)has_out_variantz*at::native::_weight_int4pack_mm_cpu_tensor)rkernel_creatorcb[R"[R[R [R /5 [R"[R5 [R"[R 5 [R"[R 5 gN)radd_needs_realized_inputs quantized max_pool2d _quantized$wrapped_fbgemm_pack_gemm_matrix_fp16!wrapped_fbgemm_linear_fp16_weight make_fallback`/home/mande/repo/quber/.venv/lib/python3.13/site-packages/torch/_inductor/quantized_lowerings.pyregister_quantized_opsr"'sn &&   ; ;  8 8  9//0 :JJK :GGHr c [[RSS9SS.S[RS[RS[RS[ S[ 4 Sjj5n[[R SS9SS.S[RS[RS [S [RS[ S[ 4 S jj5n[R"[R5 [R"[R5 g) N)type_promotion_kind)layoutinputweightscaler%returnc^^[XTSS9u nmpVUR5[R[R[R 4;a"UR5[R :XdeTn[5(a[RXVT4U5/O/nS[RS[4UU4Sjjn [XuUSS9(a[R"UUXVT/SU S9 [SXUT/U5$) NT)r%mat2_transposedbufr)c T>[US[[TTR55S9$)Nmul)other)r rr size)r,r%r(s r! _mul_epilogue?register_woq_mm_ops..int8pack_mm.._mul_epilogueOs',U.v{{1K"L r )r+)trans_wepilogue_creator_weight_int8pack_mm)r get_dtypetorchbfloat16float16floatint8raten__weight_int8pack_mmbindTensorrrr add_choicesr) r&r'r(r%_mat1mat2 aten_layoutchoicesr1s `` r! int8pack_mm(register_woq_mm_ops..int8pack_mm5s'. &$' #1a NN  L L EJJ.  / %&&& * *D+> L M  u||    !D$ O  ' 'U#!.  ) !74,?  r qGroupSizeqScaleAndZerosc X[XUSSS9u pTpgUR5[R[R[R 4;a"UR5[R :Xde[RR[R"U[RS9SS9nUn [5(a[RXgX4U 5/O/n [R (d[R"(aR[%U UUSSUS9(a?UR'5R)5(a[*UR-U U XgX/5 S[R.R0R2S[R44Sjn U S S .n [7S U XgX/U U S 9$) NT)r% use_4x2_dimr+)dtype)name)r+ is_woq_int4 q_group_sizexr)cUR5R5(deUR5nUR5n[R "SSU[R US9$)Nr)rKdevice) get_layout is_contiguousget_size get_devicer7randintuint8)rOshaperRs r!get_example_weightHregister_woq_mm_ops..int4pack_mm_cpu..get_example_weightsN<<>//11 11JJLE\\^F==Cekk&Q Qr cX[RRUR5$r)rgraph constantsget_name)rOs r!>register_woq_mm_ops..int4pack_mm_cpu..s**1::<8r )r_weight_int4pack_mm_for_cpu) input_gen_fns)rr6r7r8r9r:rXrr]add_tensor_constanttensorint64raten__weight_int4pack_mm_cpur=r max_autotunemax_autotune_gemmrrSrTr r? _inductorirIRNoder>r) r&r'rGrHr%r@rArB group_sizerCrDrZrds r!int4pack_mm_cpu,register_woq_mm_ops..int4pack_mm_cpuas'. &dD' #1a NN  L L EKK/  0WW00 LL5;; 7d1  %&& -11rrcintrr_dyn_quant_matmul_4bit_dyn_quant_pack_4bit_weight)rEros r!register_woq_mm_opsru4st//TJ  ) ||)  ) ||)  ) ) K) Vt77TR D ||D  D D  D  D  D SD L 4667 4;;rs 4N8/+ @!- 8% 2 II000'..   II   YY ! !  yy~~ Iu=r