Gemm#
Domain:
ai.onnxSince version: 13
General Matrix multiplication: https://en.wikipedia.org/wiki/Basic_Linear_Algebra_Subprograms#Level_3
A’ = transpose(A) if transA else A
B’ = transpose(B) if transB else B
Compute Y = alpha * A’ * B’ + beta * C, where input tensor A has shape (M, K) or (K, M), input tensor B has shape (K, N) or (N, K), input tensor C is broadcastable to shape (M, N), and output tensor Y has shape (M, N). A will be transposed before doing the computation if attribute transA is non-zero, same for B and transB.
Inputs
A (T): Input tensor A. The shape of A should be (M, K) if transA is 0, or (K, M) if transA is non-zero.
B (T): Input tensor B. The shape of B should be (K, N) if transB is 0, or (N, K) if transB is non-zero.
C (T): Optional input tensor C. If not specified, the computation is done as if C is a scalar 0. The shape of C should be unidirectional broadcastable to (M, N).
Outputs
Y (T): Output tensor of shape (M, N).
Type Constraints
T: Constrain input and output types to float/int tensors. Allowed types: tensor(bfloat16), tensor(double), tensor(float), tensor(float16), tensor(int32), tensor(int64), tensor(uint32), tensor(uint64).
Examples#
test_cc_gemm_all_attributes
Node:
Gemm(A, B, C) -> (Y)
Attributes:
transA = 1
transB = 1
alpha = 0.5
beta = 2.0
Inputs:
A: shape=(5, 3), dtype=float32
[[ 0.37463 , 1.1417899 , -2.034389 ],
[ 1.7053852 , -0.5352162 , 0.5568275 ],
[ 0.4316686 , 1.1318241 , 0.29380032],
[-0.45417047, 0.1968969 , -0.58087045],
[-1.4071889 , -0.8136179 , 2.617363 ]]
B: shape=(4, 5), dtype=float32
[[-0.2963621 , 1.7526379 , -0.13178441, -1.551783 , -2.1273882 ],
[-1.0876489 , -0.7198908 , 0.23265179, 0.36193573, -0.19941197],
[ 1.1704445 , 0.30147216, -0.97132784, 0.84919804, 1.3891511 ],
[-0.1005749 , 1.0460621 , 0.3995011 , 0.4361414 , -1.0567824 ]]
C: shape=(3, 4), dtype=float32
[[ 0.35325032, -0.96514314, -0.8884567 , -0.8081936 ],
[-0.15613088, 0.9437549 , 0.4275503 , 0.98345447],
[-1.3303409 , -0.00607444, 0.00345446, -0.595678 ]]
Outputs:
Y: shape=(3, 4), dtype=float32
[[ 3.9662108 , -2.6395357 , -2.6804938 , -0.01252604],
[-0.31238198, 1.7076402 , 0.41142204, 2.328485 ],
[-4.2240057 , 0.5618651 , 0.32890418, -2.2487893 ]]
test_cc_gemm_alpha
Node:
Gemm(A, B, C) -> (Y)
Attributes:
alpha = 0.5
Inputs:
A: shape=(3, 5), dtype=float32
[[ 0.8741822 , 0.553631 , -1.1274734 , -0.8376671 , -0.40274566],
[-1.941035 , 0.03275626, -0.18707356, 0.6395014 , -0.72187763],
[ 1.4065468 , -2.193512 , -1.1618237 , 0.2559442 , 2.2077847 ]]
B: shape=(5, 4), dtype=float32
[[ 1.4420623 , -0.9769679 , 0.03334304, -0.0760201 ],
[-0.9492963 , -1.0993737 , -0.20126313, 0.6861592 ],
[ 0.06786127, -0.3632581 , -0.2364485 , -0.53106284],
[ 0.4120935 , 0.0540428 , -0.68499637, -0.44563445],
[-1.0015494 , -0.00249182, 0.5274406 , -0.9984743 ]]
C: shape=(1, 4), dtype=float32
[[0., 0., 0., 0.]]
Outputs:
Y: shape=(3, 4), dtype=float32
[[ 0.35836303, -0.5486989 , 0.27284315, 0.84380364],
[-0.92817676, 0.9823163 , -0.42294157, 0.35258698],
[ 0.96302307, 0.7338553 , 0.8761188 , -1.6567483 ]]
test_cc_gemm_beta
Node:
Gemm(A, B, C) -> (Y)
Attributes:
beta = 0.5
Inputs:
A: shape=(2, 7), dtype=float32
[[-0.24912022, 0.37703994, -0.69417447, -0.7060292 , 0.20510858, -0.6286968 ,
1.6711484 ],
[ 0.33107895, 0.7037558 , -1.3299477 , 1.2570701 , 1.9146868 , -0.6062891 ,
-1.0759572 ]]
B: shape=(7, 4), dtype=float32
[[-1.313993 , -0.9773617 , -0.6343613 , -0.979214 ],
[-0.58685726, 1.6898878 , 0.08713032, 1.1718034 ],
[ 0.79937506, 0.02316316, 0.29876754, 0.3574321 ],
[ 1.1932331 , -0.9904888 , 0.60881287, -0.48373404],
[ 0.70216364, -0.8474465 , -1.2743044 , 0.29819134],
[-0.4808629 , 0.29299355, -0.61132276, 0.3887899 ],
[-1.2674389 , -0.60134256, -0.5235882 , 0.97903776]]
C: shape=(1, 4), dtype=float32
[[-0.77932763, -0.11304837, -0.02405588, -0.74120575]]
Outputs:
Y: shape=(2, 4), dtype=float32
[[-3.352695 , 0.14439091, -1.2104082 , 1.861414 ],
[ 2.1988225 , -1.6199738 , -1.2986548 , -1.6717689 ]]
test_cc_gemm_default
Node:
Gemm(A, B) -> (Y)
Inputs:
A: shape=(3, 4), dtype=float32
[[ 1.4243481 , -0.61890423, -0.5907667 , 1.4329695 ],
[ 0.5837956 , -1.3854368 , 1.2791865 , 0.32735094],
[ 0.6038593 , 0.24222691, -0.89236253, -1.1303798 ]]
B: shape=(4, 3), dtype=float32
[[ 0.5802847 , -1.4953256 , 0.42665341],
[ 0.16854794, -0.7854995 , -0.3418555 ],
[-0.1149836 , 0.95487213, 1.3025413 ],
[ 0.5168173 , -0.6690302 , 0.806894 ]]
Outputs:
Y: shape=(3, 3), dtype=float32
[[ 1.5307243 , -3.1665215 , 1.2060351 ],
[ 0.12735032, 1.2177472 , 2.6530282 ],
[-0.09035558, -1.1890693 , -1.8996036 ]]
test_cc_gemm_default_matrix_bias
Node:
Gemm(A, B, C) -> (Y)
Inputs:
A: shape=(3, 4), dtype=float32
[[ 0.16602525, 0.11929144, 0.5103543 , -0.07833039],
[-0.2179262 , -0.35897505, -0.9920161 , 1.1369699 ],
[ 0.15822072, -0.66695 , 0.1640578 , 0.5148549 ]]
B: shape=(4, 3), dtype=float32
[[ 1.2357849 , -0.33475664, 1.0486782 ],
[ 0.70084286, 0.7489813 , -1.2666165 ],
[-0.857871 , -0.38804862, -0.49571487],
[ 0.13281819, -1.4051843 , 0.05890435]]
C: shape=(3, 3), dtype=float32
[[-0.846573 , 1.2833743 , 0.7443729 ],
[-0.53892165, -0.18734488, 0.12178429],
[-2.0982637 , 1.5225194 , 0.8434675 ]]
Outputs:
Y: shape=(3, 3), dtype=float32
[[-1.0060188 , 1.2291696 , 0.5097792 ],
[-0.05778456, -1.59596 , 0.9066632 ],
[-2.4425225 , 0.18289232, 1.8031613 ]]
test_cc_gemm_default_no_bias
Node:
Gemm(A, B) -> (Y)
Inputs:
A: shape=(2, 10), dtype=float32
[[-0.7513602 , 0.51713336, 0.7695295 , -0.22347532, 0.18149193, 0.24913743,
1.994417 , -0.07321609, 0.1076815 , 2.7155633 ],
[-0.47996232, 1.631689 , 0.49418923, -2.1702976 , 2.58957 , 0.14372788,
0.06502817, 1.441035 , -0.6060344 , 1.2044882 ]]
B: shape=(10, 3), dtype=float32
[[-1.0029005 , 1.0619555 , -0.30622792],
[-0.5332439 , 1.4375478 , 1.2071373 ],
[-0.6458864 , 0.8300795 , 0.21273202],
[-0.7806308 , -0.40428525, 0.6376805 ],
[ 1.1788383 , -0.4927185 , 2.5766451 ],
[-1.7393267 , -1.0279831 , -0.6508921 ],
[-2.7996702 , 0.2743459 , 0.78074443],
[ 1.0220166 , -0.2684288 , 0.6771564 ],
[-0.21049254, 0.6646813 , -1.1061122 ],
[ 1.5222495 , -2.4748578 , 0.76601636]]
Outputs:
Y: shape=(2, 3), dtype=float32
[[-1.6116166, -5.753168 , 4.6496243],
[ 7.040774 , -2.0528429, 10.036253 ]]
test_cc_gemm_default_scalar_bias
Node:
Gemm(A, B, C) -> (Y)
Inputs:
A: shape=(2, 3), dtype=float32
[[ 0.5481833 , -0.00939301, -0.45271957],
[-1.0654595 , -0.32869837, -1.047124 ]]
B: shape=(3, 4), dtype=float32
[[ 0.6912245 , 0.83742934, -0.78804463, 0.6222988 ],
[-0.937902 , -0.7905085 , -1.8190296 , -0.00926822],
[-0.08620772, -1.7776535 , -1.2516913 , -2.1189528 ]]
C: shape=(), dtype=float32
3.14
Outputs:
Y: shape=(2, 4), dtype=float32
[[3.5667555, 4.4112687, 3.2917585, 4.4405127],
[2.8020854, 4.3690157, 5.888218 , 4.6988187]]
test_cc_gemm_default_single_elem_vector_bias
Node:
Gemm(A, B, C) -> (Y)
Inputs:
A: shape=(3, 7), dtype=float32
[[-2.4143047 , -0.6895737 , -0.16998304, 1.3226604 , -0.18856247, -0.05879052,
0.8175494 ],
[-1.1248766 , 0.33944 , -1.4056263 , 2.3914292 , 0.31966904, -0.9137981 ,
-0.6387391 ],
[ 0.25674546, -0.19762619, -0.23839723, -0.36747596, -0.05023003, 1.9321263 ,
0.09895403]]
B: shape=(7, 3), dtype=float32
[[-0.04804087, -0.30510044, -1.6544707 ],
[-0.07651462, 0.15347514, -0.8623296 ],
[ 1.2285718 , -1.4362634 , 0.62051475],
[ 0.46972257, -2.3047967 , 0.39351684],
[ 0.8641912 , -0.8612014 , -0.6527527 ],
[ 0.19660589, 0.5224583 , -0.7622916 ],
[-0.81308407, -0.2525546 , 0.09045612]]
C: shape=(1,), dtype=float32
[1.7542539]
Outputs:
Y: shape=(3, 3), dtype=float32
[[ 1.4961997 , -0.49409723, 7.000155 ],
[ 1.7946631 , -1.934763 , 3.8216138 ],
[ 1.5475421 , 3.862671 , -0.2237469 ]]
test_cc_gemm_default_vector_bias
Node:
Gemm(A, B, C) -> (Y)
Inputs:
A: shape=(2, 7), dtype=float32
[[-1.233148 , 0.13186626, 0.7939877 , 0.4511548 , 0.17302549, -1.5203902 ,
0.49947104],
[-0.45518056, -0.561509 , -0.2619732 , 0.7643679 , -0.5008771 , -1.6715151 ,
-1.0876864 ]]
B: shape=(7, 4), dtype=float32
[[-1.0009542 , 1.3335403 , 0.51886207, -0.87937015],
[-0.34444883, 0.7261377 , 0.44689482, -0.11980146],
[ 1.3952885 , 1.7246119 , 1.1329428 , -0.75994855],
[ 1.0185796 , -1.1640126 , -1.4462589 , 1.9671117 ],
[-0.78563446, -0.6370028 , -0.6990264 , -0.05147265],
[-0.93405545, -1.5452816 , -0.27895346, -0.28968734],
[ 1.3620633 , 0.25097892, 0.15541512, -0.5576033 ]]
C: shape=(4,), dtype=float32
[ 0.08745956, -1.0263144 , -0.35410684, -0.36685738]
Outputs:
Y: shape=(2, 4), dtype=float32
[[ 4.808247 , 0.63372695, -0.30716014, 1.1388453 ],
[ 1.6228232 , -0.7535494 , -1.5961344 , 2.9198632 ]]
test_cc_gemm_default_zero_bias
Node:
Gemm(A, B, C) -> (Y)
Inputs:
A: shape=(3, 5), dtype=float32
[[-0.12356076, -0.15909512, -0.641816 , -1.0080488 , -1.529806 ],
[-0.00586658, 0.07012624, 0.18260616, 0.7011332 , 0.39704362],
[-0.9507865 , 0.17773499, 1.4164393 , 0.3694542 , 0.36353227]]
B: shape=(5, 4), dtype=float32
[[-2.7012961e+00, -5.5849600e-01, 5.7748574e-01, 6.4741242e-01],
[-7.0523366e-02, -1.1587335e+00, -1.4418545e-01, 8.6498004e-01],
[ 3.7774590e-01, 3.4641349e-01, 4.6990660e-04, -1.5549998e+00],
[-2.0975480e+00, 1.6738971e-01, 2.1831582e+00, -5.0491732e-01],
[-3.2653490e-01, 1.5034325e-01, -2.0524660e-01, 7.2178906e-01]]
C: shape=(1, 4), dtype=float32
[[0., 0., 0., 0.]]
Outputs:
Y: shape=(3, 4), dtype=float32
[[ 2.7165165 , -0.3677097 , -1.9354594 , 0.18519902],
[-1.5204285 , 0.16233139, 1.4357796 , -0.2945254 ],
[ 2.1972218 , 0.93223405, 0.15793638, -2.5885265 ]]
test_cc_gemm_transposeA
Node:
Gemm(A, B) -> (Y)
Attributes:
transA = 1
Inputs:
A: shape=(4, 3), dtype=float32
[[ 0.30241716, 0.6078371 , -0.05694431],
[-0.7788967 , 0.3255947 , -0.81053257],
[ 1.0069923 , -0.6940336 , 0.5060027 ],
[-0.465687 , 0.32813475, 0.05429814]]
B: shape=(4, 5), dtype=float32
[[ 9.1375333e-01, -1.1910629e-03, -2.3453010e-02, -1.2236447e+00,
1.0235803e-01],
[ 2.6647973e-01, 9.3602963e-02, -2.5494930e-01, 2.3220780e-01,
2.7332199e-01],
[ 1.4086610e-01, 4.0873098e-01, 1.0699291e+00, 7.7722573e-01,
8.2415843e-01],
[-3.5929963e-01, 1.0196435e-01, -1.0034709e+00, 1.1876976e+00,
5.4739445e-01]]
Outputs:
Y: shape=(3, 5), dtype=float32
[[ 0.37794673, 0.29083827, 1.7362005 , -0.321352 , 0.39307198],
[ 0.42651308, -0.22046235, -1.1691062 , -0.8178669 , -0.24116534],
[-0.21625422, 0.13655506, 0.6948806 , 0.33923572, 0.21938382]]
test_cc_gemm_transposeB
Node:
Gemm(A, B) -> (Y)
Attributes:
transB = 1
Inputs:
A: shape=(3, 5), dtype=float32
[[-0.96536314, 2.3139198 , 1.874632 , -0.1776345 , 1.7384709 ],
[ 0.94759107, 0.00794787, 0.7420544 , -0.38146827, 0.55666405],
[-0.37199655, 0.12691163, -0.41666234, 0.5909262 , -0.25647718]]
B: shape=(4, 5), dtype=float32
[[ 0.7633227 , 0.76284015, 0.95591694, -0.28662303, -0.18815368],
[-1.463605 , -0.5485609 , 0.07462305, -0.04427162, 0.78086376],
[-0.4645816 , -0.7657007 , -1.3159432 , 0.0717006 , -1.287891 ],
[-0.5964969 , 1.2275231 , -0.840618 , -1.7725532 , -0.15299344]]
Outputs:
Y: shape=(3, 4), dtype=float32
[[ 2.544074 , 1.6488482 , -6.0418873 , 1.8892688 ],
[ 1.4433223 , -0.8843175 , -2.1670947 , -0.58825636],
[-0.70655066, 0.21730964, 0.99663484, -0.28027317]]
Differences with previous version (11)#
SchemaDiff: Gemm (domain 'ai.onnx')
old version: 11
new version: 13
breaking: no
Type constraints:
changed ‘T’: added types: [‘tensor(bfloat16)’]