This repository was archived by the owner on Jan 21, 2025. It is now read-only.

Commits

Commits on Nov 17, 2023

Migrate references and remove legacy target tpu:tpu_estimator.

jam14j
authored and
Mesh TensorFlow Team
committed

Commits on May 16, 2023

feat(ci): enable `pip` caching in CI (#388)
SauravMaheshkar
authored

Commits on Jan 25, 2023

Remove legacy references from `ops.py`.

jam14j
authored and
Mesh TensorFlow Team
committed

Commits on Oct 4, 2022

Fix docstring typos
Mesh TensorFlow Team
committed

Commits on Sep 19, 2022

Enable multi-file inference
Mesh TensorFlow Team
committed

Commits on Jun 10, 2022

Internal change
Mesh TensorFlow Team
committed

Commits on May 12, 2022

Make mesh_tensorflow's call of `get_replicated_var_handle` backward-compatible with tf <= 2.8.0. Fixes https://github.com/google-research/text-to-text-transfer-transformer/issues/1020.

adarob
authored and
Mesh TensorFlow Team
committed

Commits on May 5, 2022

bump version number to release updated PyPI package that includes last year enhancements

aboSamoor
authored and
Mesh TensorFlow Team
committed

Commits on Mar 25, 2022

Explicitly import estimator from tensorflow as a separate import instead of accessing it via tf.estimator and depend on the tensorflow estimator target.

qlzh727
authored and
Mesh TensorFlow Team
committed

Commits on Mar 24, 2022

Remove unused comments related to Python 2 compatibility.

yilei
authored and
Mesh TensorFlow Team
committed

Commits on Mar 21, 2022

Make TPU variable name deterministic.

chuanhaozhuge
authored and
Mesh TensorFlow Team
committed

Commits on Feb 18, 2022

#HyperPrompt Part 2 of HyperPrompt implementation: the actual computation of HyperPrompt inside self-attention layer.
Mesh TensorFlow Team
committed

Commits on Jan 26, 2022

Use math.gcd instead of fractions.gcd, the former is deprecated in Python 3.5 and removed in 3.9.

yilei
authored and
Mesh TensorFlow Team
committed
Split out optimizer call for internal purposes.

William Fedus
authored and
Mesh TensorFlow Team
committed

Commits on Jan 19, 2022

fix typo in logging statement.
Mesh TensorFlow Team
committed

Commits on Oct 18, 2021

Make sure gates are not normalized for n=1 for top_n routing
Mesh TensorFlow Team
committed

Commits on Oct 6, 2021

Add a link to the Primer paper
Mesh TensorFlow Team
committed

Commits on Sep 14, 2021

Add utility to save score predictions to TFRecords for scoring large datasets.

Hyperparticle
authored and
Mesh TensorFlow Team
committed

Commits on Sep 12, 2021

Remove unnecessary name and cwise in squared relu.

David So
authored and
Mesh TensorFlow Team
committed

Commits on Sep 10, 2021

Squared ReLU from Primer paper.

David So
authored and
Mesh TensorFlow Team
committed

Commits on Sep 8, 2021

Expert Attention Fixes:
Mesh TensorFlow Team
committed
Internal
Mesh TensorFlow Team
committed

Commits on Sep 7, 2021

Expert Attention Fixes:
Mesh TensorFlow Team
committed

Commits on Aug 24, 2021

Add more extensive top-2 logging.
Mesh TensorFlow Team
committed

Commits on Aug 20, 2021

Remove dataset checkpoint policy override now that b/181765832 is resolved.

adarob
authored and
Mesh TensorFlow Team
committed

Commits on Aug 18, 2021

heterogeneous mixture of experts layer
Mesh TensorFlow Team
committed
Only add z_loss to losses if during training.
Mesh TensorFlow Team
committed

Commits on Aug 16, 2021

Fix bug in shared_kv attention for autoregressive decoding.
Mesh TensorFlow Team
committed
Change second d_model_split dim's size to be the output shape, instead of input shape. This allows it to work for layers where the input size is different than the output size.
Mesh TensorFlow Team
committed

Commits on Aug 11, 2021

Update mesh tensorflow to use device assignments to map logical to physical processor numbers on N-D Meshes. Currently only enabled when logical cores per replica is set to 1.
Mesh TensorFlow Team
committed

Commits on Aug 10, 2021

Add more options to Experts Attention. These options remove 1/3 of the all2all communication costs:
Mesh TensorFlow Team
committed

Commits on Aug 3, 2021

Add in Z-loss to all routing algorithms.
Mesh TensorFlow Team
committed

Commits on Aug 2, 2021

Minor changes to make Experts Attention work.
Mesh TensorFlow Team
committed

Commits on Jul 29, 2021

Add top-n routing, which generalized top-2 routing. Improves model quality for larger capacity factors (e.g. 2.0+).
Mesh TensorFlow Team
committed

Commits on Jul 27, 2021

Add z_loss on all attention logits. This does not change model quality and can effectively decrease the attention logits by order of magnitude.
Mesh TensorFlow Team
committed