<p>Short of a lexicographic analysis, you can't. There is a little punctuation in Chinese, so you can split sentences into subgroups. But after that, there's no easy way to say where a word stops.</p>

<p>Even very common words like 的 (genitive marker, appears right after the possessor, eg 我的朋友: me, I; genitive; friend: my friend) cannot be relied on 100% to be always just that. 'Taxi' is written 的士, and in this case 的 would not be a single character delimiting two words, but the first character of a word.</p>

<p>And Chinese is not the only language that doesn't have spaces. You'll run into more problems like this.</p>

CC BY-SA 3.0
2015-12-02T06:55:01.483Z
2015-12-02T06:55:01.483Z
dda
<p>I am doing some work with scanning Chinese text with my computer and I have run into a problem that I do not have with other languages.</p>

<p>Because there are no spaces between characters, I cannot tell when one word ends and another one begins. Is there any easy way to do this? For example in English I am telling my computer to split things like this:</p>

<pre><code>sentence = "My name is John"
the computer will return a list of words.
computer = ['my', 'name', 'is', 'John']
</code></pre>

CC BY-SA 3.0
2015-12-02T02:09:22.083Z
No easy way, no. Sometimes a character can belong to the character before it, be on its own, or belong to the character after it, and it will change the meaning ...
Ming
CC BY-SA 3.0
2015-12-02T02:31:36.920Z
You'd probably need to match it against some sort of database.
Mou某
CC BY-SA 3.0
2015-12-02T06:53:55.273Z
Chinese word segmentation (中文分词) is a difficult problem in the field of natural language processing (NLP) that has been studying for a long time. There're a few solutions such as [Stanford Word Segmenter](http://nlp.stanford.edu/software/segmenter.shtml), [MMSEG](http://technology.chtsai.org/mmseg/), and [Sphhinx](http://sphinxsearch.com).
Stan
<characters><sentence-structure>
How to tell when one word starts and another word ends?
# How to tell when one word starts and another word ends?

(Asked by: username_0 on 2015-12-02)

I am doing some work with scanning Chinese text with my computer and I have run into a problem that I do not have with other languages.
Because there are no spaces between characters, I cannot tell when one word ends and another one begins. Is there any easy way to do this? For example in English I am telling my computer to split things like this:
sentence = "My name is John"
the computer will return a list of words.
computer = ['my', 'name', 'is', 'John']

**Tags:** characters, sentence-structure
> **Comments:**
> **username_1**: No easy way, no. Sometimes a character can belong to the character before it, be on its own, or belong to the character after it, and it will change the meaning ...  
> **username_2**: You'd probably need to match it against some sort of database.  
> **username_3**: Chinese word segmentation (中文分词) is a difficult problem in the field of natural language processing (NLP) that has been studying for a long time. There're a few solutions such as [Stanford Word Segmenter](http://nlp.stanford.edu/software/segmenter.shtml), [MMSEG](http://technology.chtsai.org/mmseg/), and [Sphhinx](http://sphinxsearch.com).  

---

## 1 answers

### Answer by username_4 on 2015-12-02. Score: 4

Short of a lexicographic analysis, you can't. There is a little punctuation in Chinese, so you can split sentences into subgroups. But after that, there's no easy way to say where a word stops.

Even very common words like 的 (genitive marker, appears right after the possessor, eg 我的朋友: me, I; genitive; friend: my friend) cannot be relied on 100% to be always just that. 'Taxi' is written 的士, and in this case 的 would not be a single character delimiting two words, but the first character of a word.

And Chinese is not the only language that doesn't have spaces. You'll run into more problems like this.


---

