Skip to content

Commit

Permalink
Resolved conflict by rebase
Browse files Browse the repository at this point in the history
  • Loading branch information
nileshkokane01 committed Feb 6, 2024
1 parent bdc243d commit 90d2459
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion src/transformers/models/blip/modeling_tf_blip_text.py
Original file line number Diff line number Diff line change
Expand Up @@ -1063,7 +1063,7 @@ def call(
# Keras won't give us label smoothing for sparse CE, so we de-sparsify things here
# Use relu to clamp masked labels at 0 to avoid NaN (we will be zeroing those out later anyway)
one_hot_labels = tf.one_hot(tf.nn.relu(labels), depth=self.config.vocab_size, dtype=tf.float32)
loss_fct = keras.losses.CategoricalCrossentropy(from_logits=True, label_smoothing=0.1, reduction="none")
loss_fct = keras.losses.CategoricalCrossentropy(from_logits=True, label_smoothing=0.0, reduction="none")
masked_positions = tf.cast(tf.not_equal(labels, -100), dtype=tf.float32)
lm_loss = loss_fct(one_hot_labels, shifted_prediction_scores)
lm_loss *= masked_positions
Expand Down

0 comments on commit 90d2459

Please sign in to comment.