RedRayz/hikari_chenkin
Updated
•
4
Nitesh Kumar Jha
nitesh501
·
AI & ML interests
None yet
Recent Activity
liked
a model
1 day ago
RedRayz/hikari_chenkin
liked
a Space
5 days ago
yuntian-deng/ChatGPT
liked
a Space
5 days ago
Alpha-VLLM/Lumina-Image-2.0
Organizations
None yet
liked
a
model
1 day ago
liked
2
Spaces
5 days ago
liked
a
Space
6 days ago
liked
2
Spaces
7 days ago
liked
2
Spaces
about 2 months ago
liked
9
Spaces
2 months ago
Demucs Music Source Separation (v4)
⚡
257
Separate vocals from instrumental music
DiffRhythm2
🎵
41
Generate music from lyrics and prompts
Song Generation
🎵
616
Generate custom songs from lyrics and prompts
Dream-wan2-2-faster-Pro
🎥
1.23k
generate a video from an image with a text prompt
Z Image Turbo
🏃
1.68k
Generate realistic images from text descriptions
Wan2.2 14B Fast
🎥
2.76k
generate a video from an image with a text prompt
Finegrain Image Enhancer
🖼
1.99k
Clarity AI Upscaler Reproduction
LongCat Video
😼
141
Generate videos from text or images
WaifuDiffusion Tagger
💬
220
Tag images with character and general tags
reacted to
Kseniase's
post with 🤗
3 months ago
Post
4125
7+ Main precision formats used in AI:
Precision is very important in AI as it shapes how accurate and efficient models are. It controls how finely numbers are represented, approximating real-world values with formats like fixed-point and floating-point. A recent BF16 → FP16 study renewed attention to precision impact.
Here are the main precision types used in AI, from full precision for training to ultra-low precision for inference:
1. FP32 (Float32):
Standard full-precision float used in most training: 1 sign bit, 8 exponent bits, 23 mantissa bits. Default for backward-compatible training and baseline numerical stability
2. FP16 (Float16) → https://arxiv.org/abs/2305.10947v6
Half-precision float. It balances accuracy and efficiency. 1 sign bit, 5 exponent bits, 10 mantissa bits. Common on NVIDIA Tensor Cores and mixed-precision setups. There’s now a new wave of using it in reinforcement learning: https://www.turingpost.com/p/fp16
3. BF16 (BFloat16) → https://cloud.google.com/blog/products/ai-machine-learning/bfloat16-the-secret-to-high-performance-on-cloud-tpus
Same dynamic range as FP32 but fewer mantissa bits: 1 sign bit, 8 exponent bits (same as FP32), 7 mantissa bits. It was developed by the research group Google Brain as part of their AI/ML infrastructure work at Google. Preferred on TPUs and modern GPUs
4. FP8 (E4M3 / E5M2) → https://proceedings.neurips.cc/paper_files/paper/2018/file/335d3d1cd7ef05ec77714a215134914c-Paper.pdf
Emerging standard for training and inference on NVIDIA Hopper (H100) and Blackwell (B200) tensor cores and AMD MI300. Also supported in NVIDIA’s Transformer Engine: https://developer.nvidia.com/blog/floating-point-8-an-introduction-to-efficient-lower-precision-ai-training/
E4M3 = 4 exponent, 3 mantissa bits
E5M2 = 5 exponent, 2 mantissa bits
Read further below ⬇️
If you like this, also subscribe to the Turing post: https://www.turingpost.com/subscribe
Precision is very important in AI as it shapes how accurate and efficient models are. It controls how finely numbers are represented, approximating real-world values with formats like fixed-point and floating-point. A recent BF16 → FP16 study renewed attention to precision impact.
Here are the main precision types used in AI, from full precision for training to ultra-low precision for inference:
1. FP32 (Float32):
Standard full-precision float used in most training: 1 sign bit, 8 exponent bits, 23 mantissa bits. Default for backward-compatible training and baseline numerical stability
2. FP16 (Float16) → https://arxiv.org/abs/2305.10947v6
Half-precision float. It balances accuracy and efficiency. 1 sign bit, 5 exponent bits, 10 mantissa bits. Common on NVIDIA Tensor Cores and mixed-precision setups. There’s now a new wave of using it in reinforcement learning: https://www.turingpost.com/p/fp16
3. BF16 (BFloat16) → https://cloud.google.com/blog/products/ai-machine-learning/bfloat16-the-secret-to-high-performance-on-cloud-tpus
Same dynamic range as FP32 but fewer mantissa bits: 1 sign bit, 8 exponent bits (same as FP32), 7 mantissa bits. It was developed by the research group Google Brain as part of their AI/ML infrastructure work at Google. Preferred on TPUs and modern GPUs
4. FP8 (E4M3 / E5M2) → https://proceedings.neurips.cc/paper_files/paper/2018/file/335d3d1cd7ef05ec77714a215134914c-Paper.pdf
Emerging standard for training and inference on NVIDIA Hopper (H100) and Blackwell (B200) tensor cores and AMD MI300. Also supported in NVIDIA’s Transformer Engine: https://developer.nvidia.com/blog/floating-point-8-an-introduction-to-efficient-lower-precision-ai-training/
E4M3 = 4 exponent, 3 mantissa bits
E5M2 = 5 exponent, 2 mantissa bits
Read further below ⬇️
If you like this, also subscribe to the Turing post: https://www.turingpost.com/subscribe
- 1 reply
liked
a
Space
3 months ago
liked
a
model
3 months ago
