Readium
Toggle table of contents
androidJvm
Target filter
androidJvm
Switch theme
Search in API
Skip to content
Readium
readium-shared
/
org.readium.r2.shared.util.tokenizer
/
Tokenizer
Tokenizer
fun
interface
Tokenizer
<
D
,
T
>
A tokenizer splits a piece of data
D
into a list of
T
tokens.
Inheritors
ContentTokenizer
DefaultTextContentTokenizer
IcuTextTokenizer
NaiveTextTokenizer
Members
Functions
tokenize
Link copied to clipboard
abstract
fun
tokenize
(
data
:
D
)
:
List
<
T
>