facebookresearch / FBTT-Embedding

This is a Tensor Train based compression library to compress sparse embedding tables used in large-scale machine learning models such as recommendation and natural language processing. We showed this library can reduce the total model size by up to 100x in Facebook’s open sourced DLRM model while achieving same model quality. Our implementation is faster than the state-of-the-art implementations. Existing the state-of-the-art library also decompresses the whole embedding tables on the fly therefore they do not provide memory reduction during runtime of the training. Our library decompresses only the requested rows therefore can provide 10,000 times memory footprint reduction per embedding table. The library also includes a software cache to store a portion of the entries in the table in decompressed format for faster lookup and process.

Summary

	CU CUH
		Main Code: 3,176 LOC (7 files) = CU (55%) + PY (32%) + CUH (7%) + CPP (4%) Secondary code: Test: 476 LOC (1); Generated: 0 LOC (0); Build & Deploy: 0 LOC (0); Other: 170 LOC (3);
		Duplication: 30%
		File Size: 55% long (>1000 LOC), 19% short (<= 200 LOC)
		Unit Size: 57% long (>100 LOC), 32% short (<= 10 LOC)
		Conditional Complexity: 0% complex (McCabe index > 50), 36% simple (McCabe index <= 5)
		Logical Component Decomposition: primary (1 component)
		1 year old 61% of code older than 365 days 0% of code not updated in the past 365 days
		0% of code updated more than 50 times Also see temporal dependencies for files frequently changed in same commits.
		Goals: Keep the system simple and easy to change (4)

Commits Trend

Latest commit date: 2022-01-13

1 commits (30 days) 1 contributors (30 days)

Commits 1 12 Contributors 1 6 2022 2021

show commits trend per language

Commits	2022 2021 7 py cu 1 3 cu 1 cpp cuh cuh
Contributors	2022 2021 5 py cu 1 3 cu 1 cpp cuh cuh