NegativeLogLikelihoodLoss#
NegativeLogLikelihoodLoss - 13#
Version
domain: main
since_version: 13
function: False
support_level: SupportType.COMMON
shape inference: True
This version of the operator has been available since version 13.
Summary
A NegativeLogLikelihoodLoss operator computes (weighted) negative log likelihood loss. Its “input” tensor has the shape of (N, C, d1, d2, …, dk) where k >= 0. The “input” tensor contains log-probabilities for input[n, :, d_1, d_2,…, d_k] being in a class of [0, C). The operator’s “target” input tensor has the shape of (N, d1, d2, …, dk). It encodes class labels (one of C classes) or it may contain a special value (indicated by an attribute ignore_index) for N x d1 x d2 x … x dk samples. The loss value for input[n, :, d_1, d_2,…d_k] being classified as class c = target[n][d_1][d_2]…[d_k] is computed as:
loss[n][d_1][d_2]…[d_k] = -input[n][c][d_1][d_2]…[d_k].
When an optional “weight” is provided, the sample loss is calculated as:
loss[n][d_1][d_2]…[d_k] = -input[n][c][d_1][d_2]…[d_k] * weight[c].
loss is zero for the case when target-value equals ignore_index.
loss[n][d_1][d_2]…[d_k] = 0, when target[n][d_1][d_2]…[d_k] = ignore_index
If “reduction” attribute is set to “none”, the operator’s output will be the above loss with shape (N, d1, d2, …, dk). If “reduction” attribute is set to “mean” (the default attribute value), the output loss is (weight) averaged:
mean(loss), if “weight” is not provided,
or if weight is provided,
sum(loss) / sum(weight[target[n][d_1][d_2]…[d_k]]]), for all samples.
- If “reduction” attribute is set to “sum”, the output is a scalar:
sum(loss).
See also https://pytorch.org/docs/stable/nn.html#torch.nn.NLLLoss.
Example 1:
// negative log likelihood loss, “none” reduction N, C, d1 = 2, 3, 2 input = [[[1.0, 2.0], [2.0, 2.0], [3.0, 2.0]],
[[0.0, 1.0], [2.0, 2.0], [1.0, 2]]]
target = [[2, 1], [0, 2]]
loss = np.zeros((N, d1)) for n in range(N):
- for d_1 in range(d1):
c = target[n][d_1] loss[n][d_1] = -input[n][c][d_1]
// print(loss) // [[-3. -2.] // [-0. -2.]]
Example 2:
// weighted negative log likelihood loss, sum reduction N, C, d1 = 2, 3, 2 input = [[[1.0, 2.0], [2.0, 2.0], [3.0, 2.0]],
[[0.0, 1.0], [2.0, 2.0], [1.0, 2]]]
target = [[2, 1], [0, 2]] weight = [0.2, 0.3, 0.1] loss = np.zeros((N, d1)) for n in range(N):
- for d_1 in range(d1):
c = target[n][d_1] loss[n][d_1] = -input[n][c][d_1] * weight[c]
loss = np.sum(loss) // print(loss) // -1.1
Example 3:
// weighted negative log likelihood loss, mean reduction N, C, d1 = 2, 3, 2 input = [[[1.0, 2.0], [2.0, 2.0], [3.0, 2.0]],
[[0.0, 1.0], [2.0, 2.0], [1.0, 2]]]
target = [[2, 1], [0, 2]] weight = [0.2, 0.3, 0.1] loss = np.zeros((N, d1)) weight_total = 0 for n in range(N):
- for d_1 in range(d1):
c = target[n][d_1] loss[n][d_1] = -input[n][c][d_1] * weight[c] weight_total = weight_total + weight[c]
loss = np.sum(loss) / weight_total // print(loss) // -1.57
Attributes
ignore_index: Specifies a target value that is ignored and does not contribute to the input gradient. It’s an optional value.
reduction: Type of reduction to apply to loss: none, sum, mean (default). ‘none’: the output is the loss for each sample. ‘sum’: the output will be summed. ‘mean’: the sum of the output will be divided by the sum of applied weights. Default value is
'mean'
.
Inputs
Between 2 and 3 inputs.
input (heterogeneous) - T: Input tensor of shape (N, C) or (N, C, d1, d2, …, dk).
target (heterogeneous) - Tind: Target tensor of shape (N) or (N, d1, d2, …, dk). Target element value shall be in range of [0, C). If ignore_index is specified, it may have a value outside [0, C) and the target values should either be in the range [0, C) or have the value ignore_index.
weight (optional, heterogeneous) - T: Optional rescaling weight tensor. If given, it has to be a tensor of size C. Otherwise, it is treated as if having all ones.
Outputs
loss (heterogeneous) - T: The negative log likelihood loss
Type Constraints
T in ( tensor(double), tensor(float), tensor(float16) ): Constrain input, weight, and output types to floating-point tensors.
Tind in ( tensor(int32), tensor(int64) ): Constrain target to integer types
Examples
input_shape_is_NC
reduction = 'none'
node = onnx.helper.make_node(
'NegativeLogLikelihoodLoss',
inputs=['input', 'target'],
outputs=['loss'],
reduction=reduction
)
N, C = 3, 5
np.random.seed(0)
input = np.random.rand(N, C).astype(np.float32)
target = np.random.randint(0, high=C, size=(N, )).astype(np.int64)
negative_log_likelihood_loss = compute_negative_log_likelihood_loss(input, target, weight=None, reduction=reduction)
expect(node, inputs=[input, target], outputs=[negative_log_likelihood_loss],
name='test_nllloss_NC')
input_shape_is_NCd1d2
reduction = 'none'
node = onnx.helper.make_node(
'NegativeLogLikelihoodLoss',
inputs=['input', 'target'],
outputs=['loss'],
reduction=reduction
)
N, C, dim1, dim2 = 3, 5, 6, 6
np.random.seed(0)
input = np.random.rand(N, C, dim1, dim2).astype(np.float32)
target = np.random.randint(0, high=C, size=(N, dim1, dim2)).astype(np.int64)
negative_log_likelihood_loss = compute_negative_log_likelihood_loss(input, target, weight=None, reduction=reduction)
expect(node, inputs=[input, target], outputs=[negative_log_likelihood_loss],
name='test_nllloss_NCd1d2')
input_shape_is_NCd1d2_reduction_mean
reduction = 'mean'
node = onnx.helper.make_node(
'NegativeLogLikelihoodLoss',
inputs=['input', 'target'],
outputs=['loss'],
reduction=reduction
)
N, C, dim1, dim2 = 3, 5, 6, 6
np.random.seed(0)
input = np.random.rand(N, C, dim1, dim2).astype(np.float32)
target = np.random.randint(0, high=C, size=(N, dim1, dim2)).astype(np.int64)
negative_log_likelihood_loss = compute_negative_log_likelihood_loss(input, target, weight=None, reduction=reduction)
expect(node, inputs=[input, target], outputs=[negative_log_likelihood_loss],
name='test_nllloss_NCd1d2_reduction_mean')
input_shape_is_NCd1d2_reduction_sum
reduction = 'sum'
node = onnx.helper.make_node(
'NegativeLogLikelihoodLoss',
inputs=['input', 'target'],
outputs=['loss'],
reduction=reduction
)
N, C, dim1, dim2 = 3, 5, 6, 6
np.random.seed(0)
input = np.random.rand(N, C, dim1, dim2).astype(np.float32)
target = np.random.randint(0, high=C, size=(N, dim1, dim2))
negative_log_likelihood_loss = compute_negative_log_likelihood_loss(input, target, weight=None, reduction=reduction)
expect(node, inputs=[input, target], outputs=[negative_log_likelihood_loss],
name='test_nllloss_NCd1d2_reduction_sum')
input_shape_is_NCd1d2_with_weight
reduction = 'none'
node = onnx.helper.make_node(
'NegativeLogLikelihoodLoss',
inputs=['input', 'target', 'weight'],
outputs=['loss'],
reduction=reduction
)
N, C, dim1, dim2 = 3, 5, 6, 6
np.random.seed(0)
input = np.random.rand(N, C, dim1, dim2).astype(np.float32)
target = np.random.randint(0, high=C, size=(N, dim1, dim2)).astype(np.int64)
weight = np.random.rand(C).astype(np.float32)
negative_log_likelihood_loss = compute_negative_log_likelihood_loss(input, target, weight=weight, reduction=reduction)
expect(node, inputs=[input, target, weight], outputs=[negative_log_likelihood_loss],
name='test_nllloss_NCd1d2_with_weight')
input_shape_is_NCd1d2_with_weight_reduction_mean
reduction = 'mean'
node = onnx.helper.make_node(
'NegativeLogLikelihoodLoss',
inputs=['input', 'target', 'weight'],
outputs=['loss'],
reduction=reduction
)
N, C, dim1, dim2 = 3, 5, 6, 6
np.random.seed(0)
input = np.random.rand(N, C, dim1, dim2).astype(np.float32)
target = np.random.randint(0, high=C, size=(N, dim1, dim2)).astype(np.int64)
weight = np.random.rand(C).astype(np.float32)
negative_log_likelihood_loss = compute_negative_log_likelihood_loss(input, target, weight=weight, reduction=reduction)
expect(node, inputs=[input, target, weight], outputs=[negative_log_likelihood_loss],
name='test_nllloss_NCd1d2_with_weight_reduction_mean')
input_shape_is_NCd1d2_with_weight_reduction_sum
reduction = 'sum'
node = onnx.helper.make_node(
'NegativeLogLikelihoodLoss',
inputs=['input', 'target', 'weight'],
outputs=['loss'],
reduction=reduction
)
N, C, dim1, dim2 = 3, 5, 6, 6
np.random.seed(0)
input = np.random.rand(N, C, dim1, dim2).astype(np.float32)
target = np.random.randint(0, high=C, size=(N, dim1, dim2)).astype(np.int64)
weight = np.random.rand(C).astype(np.float32)
negative_log_likelihood_loss = compute_negative_log_likelihood_loss(input, target, weight=weight, reduction=reduction)
expect(node, inputs=[input, target, weight], outputs=[negative_log_likelihood_loss],
name='test_nllloss_NCd1d2_with_weight_reduction_sum')
input_shape_is_NCd1d2_with_weight_reduction_sum_ii
reduction = 'sum'
ignore_index = np.int64(0)
node = onnx.helper.make_node(
'NegativeLogLikelihoodLoss',
inputs=['input', 'target', 'weight'],
outputs=['loss'],
reduction=reduction,
ignore_index=ignore_index
)
N, C, dim1, dim2 = 3, 5, 6, 6
np.random.seed(0)
input = np.random.rand(N, C, dim1, dim2).astype(np.float32)
target = np.random.randint(0, high=C, size=(N, dim1, dim2)).astype(np.int64)
target[0][0][0] = np.int64(0)
weight = np.random.rand(C).astype(np.float32)
negative_log_likelihood_loss = compute_negative_log_likelihood_loss(input, target, weight=weight, reduction=reduction, ignore_index=ignore_index)
expect(node, inputs=[input, target, weight], outputs=[negative_log_likelihood_loss],
name='test_nllloss_NCd1d2_with_weight_reduction_sum_ii')
input_shape_is_NCd1d2_no_weight_reduction_mean_ii
reduction = 'mean'
ignore_index = np.int64(1)
node = onnx.helper.make_node(
'NegativeLogLikelihoodLoss',
inputs=['input', 'target'],
outputs=['loss'],
reduction=reduction,
ignore_index=ignore_index
)
N, C, dim1, dim2 = 3, 5, 6, 6
np.random.seed(0)
input = np.random.rand(N, C, dim1, dim2).astype(np.float32)
target = np.random.randint(0, high=C, size=(N, dim1, dim2)).astype(np.int64)
target[0][0][0] = np.int64(1)
negative_log_likelihood_loss = compute_negative_log_likelihood_loss(input, target, reduction=reduction, ignore_index=ignore_index)
expect(node, inputs=[input, target], outputs=[negative_log_likelihood_loss],
name='test_nllloss_NCd1d2_no_weight_reduction_mean_ii')
input_shape_is_NCd1
reduction = 'mean'
node = onnx.helper.make_node(
'NegativeLogLikelihoodLoss',
inputs=['input', 'target'],
outputs=['loss'],
reduction=reduction
)
N, C, d1 = 3, 5, 2
np.random.seed(0)
input = np.random.rand(N, C, d1).astype(np.float32)
target = np.random.randint(0, high=C, size=(N, d1)).astype(np.int64)
negative_log_likelihood_loss = compute_negative_log_likelihood_loss(input, target, weight=None, reduction=reduction)
expect(node, inputs=[input, target], outputs=[negative_log_likelihood_loss],
name='test_nllloss_NCd1')
input_shape_is_NCd1_weight
reduction = 'mean'
node = onnx.helper.make_node(
'NegativeLogLikelihoodLoss',
inputs=['input', 'target', 'weight'],
outputs=['loss'],
reduction=reduction
)
N, C, d1 = 3, 5, 2
np.random.seed(0)
input = np.random.rand(N, C, d1).astype(np.float32)
target = np.random.randint(0, high=C, size=(N, d1)).astype(np.int64)
weight = np.random.rand(C).astype(np.float32)
negative_log_likelihood_loss = compute_negative_log_likelihood_loss(input, target, weight=weight, reduction=reduction)
expect(node, inputs=[input, target, weight], outputs=[negative_log_likelihood_loss],
name='test_nllloss_NCd1_weight')
input_shape_is_NCd1_ii
reduction = 'mean'
ignore_index = np.int64(1)
node = onnx.helper.make_node(
'NegativeLogLikelihoodLoss',
inputs=['input', 'target'],
outputs=['loss'],
reduction=reduction,
ignore_index=ignore_index
)
N, C, d1 = 3, 5, 2
np.random.seed(0)
input = np.random.rand(N, C, d1).astype(np.float32)
target = np.random.randint(0, high=C, size=(N, d1)).astype(np.int64)
target[0][0] = np.int64(1)
negative_log_likelihood_loss = compute_negative_log_likelihood_loss(input, target, weight=None, reduction=reduction, ignore_index=ignore_index)
expect(node, inputs=[input, target], outputs=[negative_log_likelihood_loss],
name='test_nllloss_NCd1_ii')
input_shape_is_NCd1_weight_ii
reduction = 'mean'
ignore_index = np.int64(1)
node = onnx.helper.make_node(
'NegativeLogLikelihoodLoss',
inputs=['input', 'target', 'weight'],
outputs=['loss'],
reduction=reduction,
ignore_index=ignore_index
)
N, C, d1 = 3, 5, 2
np.random.seed(0)
input = np.random.rand(N, C, d1).astype(np.float32)
target = np.random.randint(0, high=C, size=(N, d1)).astype(np.int64)
target[0][0] = np.int64(1)
weight = np.random.rand(C).astype(np.float32)
negative_log_likelihood_loss = compute_negative_log_likelihood_loss(input, target, weight=weight, reduction=reduction, ignore_index=ignore_index)
expect(node, inputs=[input, target, weight], outputs=[negative_log_likelihood_loss],
name='test_nllloss_NCd1_weight_ii')
input_shape_is_NCd1d2d3d4d5_mean_weight
reduction = 'mean'
node = onnx.helper.make_node(
'NegativeLogLikelihoodLoss',
inputs=['input', 'target', 'weight'],
outputs=['loss'],
reduction=reduction)
N, C, dim1, dim2, dim3, dim4, dim5 = 3, 5, 6, 6, 5, 3, 4
np.random.seed(0)
input = np.random.rand(N, C, dim1, dim2, dim3, dim4, dim5).astype(np.float32)
target = np.random.randint(0, high=C, size=(N, dim1, dim2, dim3, dim4, dim5)).astype(np.int64)
weight = np.random.rand(C).astype(np.float32)
negative_log_likelihood_loss = compute_negative_log_likelihood_loss(input,
target,
weight=weight,
reduction=reduction)
expect(node, inputs=[input, target, weight], outputs=[negative_log_likelihood_loss],
name='test_nllloss_NCd1d2d3d4d5_mean_weight')
input_shape_is_NCd1d2d3d4d5_none_no_weight
reduction = 'none'
node = onnx.helper.make_node(
'NegativeLogLikelihoodLoss',
inputs=['input', 'target'],
outputs=['loss'],
reduction=reduction)
N, C, dim1, dim2, dim3, dim4, dim5 = 3, 5, 6, 6, 5, 3, 4
np.random.seed(0)
input = np.random.rand(N, C, dim1, dim2, dim3, dim4, dim5).astype(np.float32)
target = np.random.randint(0, high=C, size=(N, dim1, dim2, dim3, dim4, dim5)).astype(np.int64)
negative_log_likelihood_loss = compute_negative_log_likelihood_loss(input,
target,
reduction=reduction)
expect(node, inputs=[input, target], outputs=[negative_log_likelihood_loss],
name='test_nllloss_NCd1d2d3d4d5_none_no_weight')
input_shape_is_NCd1_mean_weight_negative_ii
reduction = 'mean'
ignore_index = np.int64(-1)
node = onnx.helper.make_node(
'NegativeLogLikelihoodLoss',
inputs=['input', 'target', 'weight'],
outputs=['loss'],
reduction=reduction,
ignore_index=ignore_index)
N, C, dim1 = 3, 5, 6
np.random.seed(0)
input = np.random.rand(N, C, dim1).astype(np.float32)
target = np.random.randint(0, high=C, size=(N, dim1)).astype(np.int64)
target[0][0] = -1
weight = np.random.rand(C).astype(np.float32)
negative_log_likelihood_loss = compute_negative_log_likelihood_loss(input,
target,
weight=weight,
reduction=reduction,
ignore_index=ignore_index)
expect(node, inputs=[input, target, weight], outputs=[negative_log_likelihood_loss],
name='test_nllloss_NCd1_mean_weight_negative_ii')
input_shape_is_NCd1d2d3_none_no_weight_negative_ii
reduction = 'none'
ignore_index = np.int64(-5)
node = onnx.helper.make_node(
'NegativeLogLikelihoodLoss',
inputs=['input', 'target'],
outputs=['loss'],
reduction=reduction,
ignore_index=ignore_index)
N, C, dim1, dim2, dim3 = 3, 5, 6, 6, 5
np.random.seed(0)
input = np.random.rand(N, C, dim1, dim2, dim3).astype(np.float32)
target = np.random.randint(0, high=C, size=(N, dim1, dim2, dim3)).astype(np.int64)
target[0][0][0][0] = -5
negative_log_likelihood_loss = compute_negative_log_likelihood_loss(input,
target,
reduction=reduction,
ignore_index=ignore_index)
expect(node, inputs=[input, target], outputs=[negative_log_likelihood_loss],
name='test_nllloss_NCd1d2d3_none_no_weight_negative_ii')
input_shape_is_NCd1d2d3_sum_weight_high_ii
reduction = 'sum'
ignore_index = np.int64(10)
node = onnx.helper.make_node(
'NegativeLogLikelihoodLoss',
inputs=['input', 'target', 'weight'],
outputs=['loss'],
reduction=reduction,
ignore_index=ignore_index)
N, C = 3, 5
np.random.seed(0)
input = np.random.rand(N, C).astype(np.float32)
target = np.random.randint(0, high=C, size=(N))
target[0] = 10
weight = np.random.rand(C).astype(np.float32)
negative_log_likelihood_loss = compute_negative_log_likelihood_loss(input,
target,
weight=weight,
reduction=reduction,
ignore_index=ignore_index)
expect(node, inputs=[input, target, weight], outputs=[negative_log_likelihood_loss],
name='test_nllloss_NCd1d2d3_sum_weight_high_ii')
Differences
0 | 0 | A NegativeLogLikelihoodLoss operator computes (weighted) negative log likelihood loss. | A NegativeLogLikelihoodLoss operator computes (weighted) negative log likelihood loss. |
1 | 1 | Its "input" tensor has the shape of (N, C, d1, d2, ..., dk) where k >= 0. | Its "input" tensor has the shape of (N, C, d1, d2, ..., dk) where k >= 0. |
2 | 2 | The "input" tensor contains log-probabilities for input[n, :, d_1, d_2,..., d_k] being in a class of [0, C). | The "input" tensor contains log-probabilities for input[n, :, d_1, d_2,..., d_k] being in a class of [0, C). |
3 | 3 | The operator's "target" input tensor has the shape of (N, d1, d2, ..., dk). It encodes class labels (one of C classes) | The operator's "target" input tensor has the shape of (N, d1, d2, ..., dk). It encodes class labels (one of C classes) |
4 | 4 | or it may contain a special value (indicated by an attribute ignore_index) for N x d1 x d2 x ... x dk samples. | or it may contain a special value (indicated by an attribute ignore_index) for N x d1 x d2 x ... x dk samples. |
5 | 5 | The loss value for input[n, :, d_1, d_2,...d_k] being classified as class c = target[n][d_1][d_2]...[d_k] is computed as: | The loss value for input[n, :, d_1, d_2,...d_k] being classified as class c = target[n][d_1][d_2]...[d_k] is computed as: |
6 |
| ||
6 | 7 | loss[n][d_1][d_2]...[d_k] = -input[n][c][d_1][d_2]...[d_k]. | loss[n][d_1][d_2]...[d_k] = -input[n][c][d_1][d_2]...[d_k]. |
8 |
| ||
7 | 9 | When an optional "weight" is provided, the sample loss is calculated as: | When an optional "weight" is provided, the sample loss is calculated as: |
10 |
| ||
8 | 11 | loss[n][d_1][d_2]...[d_k] = -input[n][c][d_1][d_2]...[d_k] * weight[c]. | loss[n][d_1][d_2]...[d_k] = -input[n][c][d_1][d_2]...[d_k] * weight[c]. |
12 |
| ||
9 | 13 | loss is zero for the case when target-value equals ignore_index. | loss is zero for the case when target-value equals ignore_index. |
10 | 14 |
|
|
11 | 15 | loss[n][d_1][d_2]...[d_k] = 0, when target[n][d_1][d_2]...[d_k] = ignore_index | loss[n][d_1][d_2]...[d_k] = 0, when target[n][d_1][d_2]...[d_k] = ignore_index |
16 |
| ||
12 | 17 | If "reduction" attribute is set to "none", the operator's output will be the above loss with shape (N, d1, d2, ..., dk). | If "reduction" attribute is set to "none", the operator's output will be the above loss with shape (N, d1, d2, ..., dk). |
13 | 18 | If "reduction" attribute is set to "mean" (the default attribute value), the output loss is (weight) averaged: | If "reduction" attribute is set to "mean" (the default attribute value), the output loss is (weight) averaged: |
19 |
| ||
14 | 20 | mean(loss), if "weight" is not provided, | mean(loss), if "weight" is not provided, |
21 |
| ||
15 | 22 | or if weight is provided, | or if weight is provided, |
23 |
| ||
16 | 24 | sum(loss) / sum(weight[target[n][d_1][d_2]...[d_k]]]), for all samples. | sum(loss) / sum(weight[target[n][d_1][d_2]...[d_k]]]), for all samples. |
25 |
| ||
17 | 26 | If "reduction" attribute is set to "sum", the output is a scalar: | If "reduction" attribute is set to "sum", the output is a scalar: |
18 | 27 | sum(loss). | sum(loss). |
28 |
| ||
19 | 29 | See also https://pytorch.org/docs/stable/nn.html#torch.nn.NLLLoss. | See also https://pytorch.org/docs/stable/nn.html#torch.nn.NLLLoss. |
30 |
| ||
20 | 31 | Example 1: | Example 1: |
32 |
| ||
21 | 33 | // negative log likelihood loss, "none" reduction | // negative log likelihood loss, "none" reduction |
22 | 34 | N, C, d1 = 2, 3, 2 | N, C, d1 = 2, 3, 2 |
23 | 35 | input = [[[1.0, 2.0], [2.0, 2.0], [3.0, 2.0]], | input = [[[1.0, 2.0], [2.0, 2.0], [3.0, 2.0]], |
24 | 36 | [[0.0, 1.0], [2.0, 2.0], [1.0, 2]]] | [[0.0, 1.0], [2.0, 2.0], [1.0, 2]]] |
25 | 37 | target = [[2, 1], [0, 2]] | target = [[2, 1], [0, 2]] |
38 |
| ||
26 | 39 | loss = np.zeros((N, d1)) | loss = np.zeros((N, d1)) |
27 | 40 | for n in range(N): | for n in range(N): |
28 | 41 | for d_1 in range(d1): | for d_1 in range(d1): |
29 | 42 | c = target[n][d_1] | c = target[n][d_1] |
30 | 43 | loss[n][d_1] = -input[n][c][d_1] | loss[n][d_1] = -input[n][c][d_1] |
44 |
| ||
31 | 45 | // print(loss) | // print(loss) |
32 | 46 | // [[-3. -2.] | // [[-3. -2.] |
33 | 47 | // [-0. -2.]] | // [-0. -2.]] |
48 |
| ||
34 | 49 | Example 2: | Example 2: |
50 |
| ||
35 | 51 | // weighted negative log likelihood loss, sum reduction | // weighted negative log likelihood loss, sum reduction |
36 | 52 | N, C, d1 = 2, 3, 2 | N, C, d1 = 2, 3, 2 |
37 | 53 | input = [[[1.0, 2.0], [2.0, 2.0], [3.0, 2.0]], | input = [[[1.0, 2.0], [2.0, 2.0], [3.0, 2.0]], |
38 | 54 | [[0.0, 1.0], [2.0, 2.0], [1.0, 2]]] | [[0.0, 1.0], [2.0, 2.0], [1.0, 2]]] |
39 | 55 | target = [[2, 1], [0, 2]] | target = [[2, 1], [0, 2]] |
40 | 56 | weight = [0.2, 0.3, 0.1] | weight = [0.2, 0.3, 0.1] |
41 | 57 | loss = np.zeros((N, d1)) | loss = np.zeros((N, d1)) |
42 | 58 | for n in range(N): | for n in range(N): |
43 | 59 | for d_1 in range(d1): | for d_1 in range(d1): |
44 | 60 | c = target[n][d_1] | c = target[n][d_1] |
45 | 61 | loss[n][d_1] = -input[n][c][d_1] * weight[c] | loss[n][d_1] = -input[n][c][d_1] * weight[c] |
62 |
| ||
46 | 63 | loss = np.sum(loss) | loss = np.sum(loss) |
47 | 64 | // print(loss) | // print(loss) |
48 | 65 | // -1.1 | // -1.1 |
66 |
| ||
49 | 67 | Example 3: | Example 3: |
68 |
| ||
50 | 69 | // weighted negative log likelihood loss, mean reduction | // weighted negative log likelihood loss, mean reduction |
51 | 70 | N, C, d1 = 2, 3, 2 | N, C, d1 = 2, 3, 2 |
52 | 71 | input = [[[1.0, 2.0], [2.0, 2.0], [3.0, 2.0]], | input = [[[1.0, 2.0], [2.0, 2.0], [3.0, 2.0]], |
53 | 72 | [[0.0, 1.0], [2.0, 2.0], [1.0, 2]]] | [[0.0, 1.0], [2.0, 2.0], [1.0, 2]]] |
54 | 73 | target = [[2, 1], [0, 2]] | target = [[2, 1], [0, 2]] |
55 | 74 | weight = [0.2, 0.3, 0.1] | weight = [0.2, 0.3, 0.1] |
56 | 75 | loss = np.zeros((N, d1)) | loss = np.zeros((N, d1)) |
57 | 76 | weight_total = 0 | weight_total = 0 |
58 | 77 | for n in range(N): | for n in range(N): |
59 | 78 | for d_1 in range(d1): | for d_1 in range(d1): |
60 | 79 | c = target[n][d_1] | c = target[n][d_1] |
61 | 80 | loss[n][d_1] = -input[n][c][d_1] * weight[c] | loss[n][d_1] = -input[n][c][d_1] * weight[c] |
62 | 81 | weight_total = weight_total + weight[c] | weight_total = weight_total + weight[c] |
82 |
| ||
63 | 83 | loss = np.sum(loss) / weight_total | loss = np.sum(loss) / weight_total |
64 | 84 | // print(loss) | // print(loss) |
65 | 85 | // -1.57 | // -1.57 |
66 | 86 |
|
|
67 | 87 | **Attributes** | **Attributes** |
68 | 88 |
|
|
69 | 89 | * **ignore_index**: | * **ignore_index**: |
70 | 90 | Specifies a target value that is ignored and does not contribute to | Specifies a target value that is ignored and does not contribute to |
71 | 91 | the input gradient. It's an optional value. | the input gradient. It's an optional value. |
72 | 92 | * **reduction**: | * **reduction**: |
73 | 93 | Type of reduction to apply to loss: none, sum, mean (default). | Type of reduction to apply to loss: none, sum, mean (default). |
74 | 94 | 'none': the output is the loss for each sample. 'sum': the output | 'none': the output is the loss for each sample. 'sum': the output |
75 | 95 | will be summed. 'mean': the sum of the output will be divided by the | will be summed. 'mean': the sum of the output will be divided by the |
76 | 96 | sum of applied weights. Default value is 'mean'. | sum of applied weights. Default value is 'mean'. |
77 | 97 |
|
|
78 | 98 | **Inputs** | **Inputs** |
79 | 99 |
|
|
80 | 100 | Between 2 and 3 inputs. | Between 2 and 3 inputs. |
81 | 101 |
|
|
82 | 102 | * **input** (heterogeneous) - **T**: | * **input** (heterogeneous) - **T**: |
83 | 103 | Input tensor of shape (N, C) or (N, C, d1, d2, ..., dk). | Input tensor of shape (N, C) or (N, C, d1, d2, ..., dk). |
84 | 104 | * **target** (heterogeneous) - **Tind**: | * **target** (heterogeneous) - **Tind**: |
85 | 105 | Target tensor of shape (N) or (N, d1, d2, ..., dk). Target element | Target tensor of shape (N) or (N, d1, d2, ..., dk). Target element |
86 | 106 | value shall be in range of [0, C). If ignore_index is specified, it | value shall be in range of [0, C). If ignore_index is specified, it |
87 | 107 | may have a value outside [0, C) and the target values should either | may have a value outside [0, C) and the target values should either |
88 | 108 | be in the range [0, C) or have the value ignore_index. | be in the range [0, C) or have the value ignore_index. |
89 | 109 | * **weight** (optional, heterogeneous) - **T**: | * **weight** (optional, heterogeneous) - **T**: |
90 | 110 | Optional rescaling weight tensor. If given, it has to be a tensor of | Optional rescaling weight tensor. If given, it has to be a tensor of |
91 | 111 | size C. Otherwise, it is treated as if having all ones. | size C. Otherwise, it is treated as if having all ones. |
92 | 112 |
|
|
93 | 113 | **Outputs** | **Outputs** |
94 | 114 |
|
|
95 | 115 | * **loss** (heterogeneous) - **T**: | * **loss** (heterogeneous) - **T**: |
96 | 116 | The negative log likelihood loss | The negative log likelihood loss |
97 | 117 |
|
|
98 | 118 | **Type Constraints** | **Type Constraints** |
99 | 119 |
|
|
100 | 120 | * **T** in ( | * **T** in ( |
101 | 121 | tensor(double), | tensor(double), |
102 | 122 | tensor(float), | tensor(float), |
103 | 123 | tensor(float16) | tensor(float16) |
104 | 124 | ): | ): |
105 | 125 | Constrain input, weight, and output types to floating-point tensors. | Constrain input, weight, and output types to floating-point tensors. |
106 | 126 | * **Tind** in ( | * **Tind** in ( |
107 | 127 | tensor(int32), | tensor(int32), |
108 | 128 | tensor(int64) | tensor(int64) |
109 | 129 | ): | ): |
110 | 130 | Constrain target to integer types | Constrain target to integer types |
NegativeLogLikelihoodLoss - 12#
Version
domain: main
since_version: 12
function: False
support_level: SupportType.COMMON
shape inference: True
This version of the operator has been available since version 12.
Summary
A NegativeLogLikelihoodLoss operator computes (weighted) negative log likelihood loss. Its “input” tensor has the shape of (N, C, d1, d2, …, dk) where k >= 0. The “input” tensor contains log-probabilities for input[n, :, d_1, d_2,…, d_k] being in a class of [0, C). The operator’s “target” input tensor has the shape of (N, d1, d2, …, dk). It encodes class labels (one of C classes) or it may contain a special value (indicated by an attribute ignore_index) for N x d1 x d2 x … x dk samples. The loss value for input[n, :, d_1, d_2,…d_k] being classified as class c = target[n][d_1][d_2]…[d_k] is computed as:
loss[n][d_1][d_2]…[d_k] = -input[n][c][d_1][d_2]…[d_k].
- When an optional “weight” is provided, the sample loss is calculated as:
loss[n][d_1][d_2]…[d_k] = -input[n][c][d_1][d_2]…[d_k] * weight[c].
loss is zero for the case when target-value equals ignore_index.
loss[n][d_1][d_2]…[d_k] = 0, when target[n][d_1][d_2]…[d_k] = ignore_index
If “reduction” attribute is set to “none”, the operator’s output will be the above loss with shape (N, d1, d2, …, dk). If “reduction” attribute is set to “mean” (the default attribute value), the output loss is (weight) averaged:
mean(loss), if “weight” is not provided,
- or if weight is provided,
sum(loss) / sum(weight[target[n][d_1][d_2]…[d_k]]]), for all samples.
- If “reduction” attribute is set to “sum”, the output is a scalar:
sum(loss).
See also https://pytorch.org/docs/stable/nn.html#torch.nn.NLLLoss. Example 1:
// negative log likelihood loss, “none” reduction N, C, d1 = 2, 3, 2 input = [[[1.0, 2.0], [2.0, 2.0], [3.0, 2.0]],
[[0.0, 1.0], [2.0, 2.0], [1.0, 2]]]
target = [[2, 1], [0, 2]] loss = np.zeros((N, d1)) for n in range(N):
- for d_1 in range(d1):
c = target[n][d_1] loss[n][d_1] = -input[n][c][d_1]
// print(loss) // [[-3. -2.] // [-0. -2.]]
- Example 2:
// weighted negative log likelihood loss, sum reduction N, C, d1 = 2, 3, 2 input = [[[1.0, 2.0], [2.0, 2.0], [3.0, 2.0]],
[[0.0, 1.0], [2.0, 2.0], [1.0, 2]]]
target = [[2, 1], [0, 2]] weight = [0.2, 0.3, 0.1] loss = np.zeros((N, d1)) for n in range(N):
- for d_1 in range(d1):
c = target[n][d_1] loss[n][d_1] = -input[n][c][d_1] * weight[c]
loss = np.sum(loss) // print(loss) // -1.1
- Example 3:
// weighted negative log likelihood loss, mean reduction N, C, d1 = 2, 3, 2 input = [[[1.0, 2.0], [2.0, 2.0], [3.0, 2.0]],
[[0.0, 1.0], [2.0, 2.0], [1.0, 2]]]
target = [[2, 1], [0, 2]] weight = [0.2, 0.3, 0.1] loss = np.zeros((N, d1)) weight_total = 0 for n in range(N):
- for d_1 in range(d1):
c = target[n][d_1] loss[n][d_1] = -input[n][c][d_1] * weight[c] weight_total = weight_total + weight[c]
loss = np.sum(loss) / weight_total // print(loss) // -1.57
Attributes
ignore_index: Specifies a target value that is ignored and does not contribute to the input gradient. It’s an optional value.
reduction: Type of reduction to apply to loss: none, sum, mean (default). ‘none’: the output is the loss for each sample. ‘sum’: the output will be summed. ‘mean’: the sum of the output will be divided by the sum of applied weights. Default value is
'mean'
.
Inputs
Between 2 and 3 inputs.
input (heterogeneous) - T: Input tensor of shape (N, C) or (N, C, d1, d2, …, dk).
target (heterogeneous) - Tind: Target tensor of shape (N) or (N, d1, d2, …, dk). Target element value shall be in range of [0, C). If ignore_index is specified, it may have a value outside [0, C) and the target values should either be in the range [0, C) or have the value ignore_index.
weight (optional, heterogeneous) - T: Optional rescaling weight tensor. If given, it has to be a tensor of size C. Otherwise, it is treated as if having all ones.
Outputs
loss (heterogeneous) - T: The negative log likelihood loss
Type Constraints
T in ( tensor(double), tensor(float), tensor(float16) ): Constrain input, weight, and output types to floating-point tensors.
Tind in ( tensor(int32), tensor(int64) ): Constrain target to integer types