2 Commits

Author	SHA1	Message	Date
Antoni Baum	dd54a4b026	Fix detokenization leaving special tokens (#1044) ... Signed-off-by: Antoni Baum <antoni.baum@protonmail.com>	2023-09-14 16:37:03 -07:00
Antoni Baum	9841d48a10	Use TGI-like incremental detokenization (#984)	2023-09-13 13:38:01 -07:00