Answers.Body: <p>As far as I know, no such project or software or site exists. The closest one I could find is (was?) a <a href="http://www.eurobricks.com/forum/index.php?showtopic=75840" rel="nofollow">thread in the Eurobircks forum</a>, where exactly this topic came up. The members there tossed about various ideas, mentioning Akiyuki's brick sorter device (with the factors that made its job easier than the one you propose: uniformly lit background with known color, precision scale, only one piece at a time, a few hundred possible parts), some proof-of-concept code was written to test the capabilities of the OpenCV library, but then the interest faded and the topic was abandoned.</p>
  
  <p>There are several problems in this proposition that are highly difficult (if not straight impossible) to overcome:</p>
  
  <ol>
  <li>Lighting conditions</li>
  </ol>
  
  <p>The optical recognition process depends heavily on sensing the difference in the brightness and color of adjacent points. If the lighting conditions are allowed to vary (that is, not requiring the use of a <a href="https://aswirly.files.wordpress.com/2009/09/more-light1.jpg" rel="nofollow">light box</a>) then the differences between each color and the background can't be set in the program. One could still overcome this limitation if the image contains a piece of a known color, but then the calibration must be done for each image.</p>
  
  <ol start="2">
  <li>Overlap</li>
  </ol>
  
  <p>If pieces are allowed to overlap then they can be easily mistaken for similar pieces, or if sufficient parts are overlapping then clear identification can become impossible. If you can only see a part of a Technic axle, you can't tell how much of it is under the 6x6 plate, so the lenght of the axle can't be determined. The same analogy works for many kinds of pieces (bricks, tiles, plates).</p>
  
  <ol start="3">
  <li>One point of view</li>
  </ol>
  
  <p>If this system can work using only a single image (taken with an ordinary camera) then some pieces can become too similar to tell apart. For example, from straight up, a 2x2 brick and a 2x2 plate look exactly the same, just like a 2x2 brick and a 3x2 brick if both are standing on their end. Shadow analysis could help with this matter, but only if there are clear shadows cast from a known point in space related to the camera and the surface on which the pieces are laid out. So we have essentially arrived at the "very precisely controlled and known lighting and photographing setup", called light box, but also added a ton of processing logic and advanced mathematics. Not to mention the problem of dealing with shadows cast on other pieces, which can render shadow analysis moot or even obscure some pieces so they won't be recognized correctly or in extreme scenarios maybe not even seen.</p>
  
  <ol start="4">
  <li>Differences only visible from one side</li>
  </ol>
  
  <p>This might not concern you, but in some cases pieces only differ from each other in a very small detail that can only be seen from a certain point of view. For example, if a minifig head was laying on its side, it is optically impossible to tell if it has a hole on the top or not.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2015-10-16T08:19:04.013Z
Answers.LastActivityDate: 2015-10-16T08:19:04.013Z
Answers.UserDisplayName: zovits
Body: <p>Is there any software/site that tells me which LEGO parts exist in a photo? I would love to get a csv of parts from a photo.</p>
  
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2015-06-20T17:44:21.327Z
Comments.Text: As far as I'm aware, there isn't such a service. [Akiyuki designed a LEGO part sorter](https://www.youtube.com/watch?v=6lZ9rSZwDzE) that sorts bricks by identifying the shape and weight of the brick. If you can get an accurate reading of a pieces weight in milligrams that should help you get a more accurate identification.
Comments.UserDisplayName: Ambo100
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2015-06-20T17:48:44.357Z
Comments.Text: [LDraw](http://www.ldraw.org/parts/latest-parts.html) distribute a library of most LEGO parts in 3D. You may also be interested in the [Signal Processing](http://dsp.stackexchange.com/) Stack Exchange site.
Comments.UserDisplayName: Ambo100
Tags: <part-identification>
Title: How can I extract part id or design id from a photo?
UserDisplayName: Peter
ThreadText: # How can I extract part id or design id from a photo?
  
  (Asked by: username_0 on 2015-06-20)
  
  Is there any software/site that tells me which LEGO parts exist in a photo? I would love to get a csv of parts from a photo.
  
  **Tags:** part-identification
  > **Comments:**
  > **username_1**: As far as I'm aware, there isn't such a service. [Akiyuki designed a LEGO part sorter](https://www.youtube.com/watch?v=6lZ9rSZwDzE) that sorts bricks by identifying the shape and weight of the brick. If you can get an accurate reading of a pieces weight in milligrams that should help you get a more accurate identification.  
  > **username_1**: [LDraw](http://www.ldraw.org/parts/latest-parts.html) distribute a library of most LEGO parts in 3D. You may also be interested in the [Signal Processing](http://dsp.stackexchange.com/) Stack Exchange site.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2015-10-16. Score: 3
  
  As far as I know, no such project or software or site exists. The closest one I could find is (was?) a [thread in the Eurobircks forum](http://www.eurobricks.com/forum/index.php?showtopic=75840), where exactly this topic came up. The members there tossed about various ideas, mentioning Akiyuki's brick sorter device (with the factors that made its job easier than the one you propose: uniformly lit background with known color, precision scale, only one piece at a time, a few hundred possible parts), some proof-of-concept code was written to test the capabilities of the OpenCV library, but then the interest faded and the topic was abandoned.
  
  There are several problems in this proposition that are highly difficult (if not straight impossible) to overcome:
  
  1. Lighting conditions
  
  The optical recognition process depends heavily on sensing the difference in the brightness and color of adjacent points. If the lighting conditions are allowed to vary (that is, not requiring the use of a [light box](https://aswirly.files.wordpress.com/2009/09/more-light1.jpg)) then the differences between each color and the background can't be set in the program. One could still overcome this limitation if the image contains a piece of a known color, but then the calibration must be done for each image.
  
  2. Overlap
  
  If pieces are allowed to overlap then they can be easily mistaken for similar pieces, or if sufficient parts are overlapping then clear identification can become impossible. If you can only see a part of a Technic axle, you can't tell how much of it is under the 6x6 plate, so the lenght of the axle can't be determined. The same analogy works for many kinds of pieces (bricks, tiles, plates).
  
  3. One point of view
  
  If this system can work using only a single image (taken with an ordinary camera) then some pieces can become too similar to tell apart. For example, from straight up, a 2x2 brick and a 2x2 plate look exactly the same, just like a 2x2 brick and a 3x2 brick if both are standing on their end. Shadow analysis could help with this matter, but only if there are clear shadows cast from a known point in space related to the camera and the surface on which the pieces are laid out. So we have essentially arrived at the "very precisely controlled and known lighting and photographing setup", called light box, but also added a ton of processing logic and advanced mathematics. Not to mention the problem of dealing with shadows cast on other pieces, which can render shadow analysis moot or even obscure some pieces so they won't be recognized correctly or in extreme scenarios maybe not even seen.
  
  4. Differences only visible from one side
  
  This might not concern you, but in some cases pieces only differ from each other in a very small detail that can only be seen from a certain point of view. For example, if a minifig head was laying on its side, it is optically impossible to tell if it has a hole on the top or not.
  
  
  ---
  
  