- Timestamp:
- 01/18/13 22:43:56 (12 years ago)
- Branches:
- master
- Children:
- b97a556
- Parents:
- 243d027
- git-author:
- Tomasz Obrebski <to@…> (01/18/13 22:43:56)
- git-committer:
- Tomasz Obrebski <to@…> (01/18/13 22:43:56)
- File:
-
- 1 edited
Legend:
- Unmodified
- Added
- Removed
-
src/tok.l/tok_cmdline.ggo
r243d027 re0cd003 12 12 13 13 tok reads from standard input, identifies tokens on the basis of their orthographic form and writes a sequence of segments in UTT format to 14 the standard output. The type of the token is printed as the type field.14 the standard output. 15 15 16 16 OUTPUT FORMAT 17 17 18 UTT-file with four fields: start, length, type, and form. In the typefield five types of tokens are distinguished:18 UTT-file with four fields: START, LENGTH, TYPE, and FORM. In the TYPE field five types of tokens are distinguished: 19 19 20 20 W (word) - continuous sequence of letters
Note: See TracChangeset
for help on using the changeset viewer.