Learn a Machine to Recognize Different Tea Types

Author/s: Paul Schmidt, Yeeun Namkoong
Posted: 17 June 2020

We decided to use different tea types for the task to familiarise with Googles Teachable Machine. We used ‘Green Tea’, ‘Matcha Tea’, ‘Black Tea’, ‘Black Tea with Milk’ and ‘Coffee with Milk’ – as the odd one out.

Overview of Data-Input

We have multiple layers of distinguishment between our classes. Green Tea and Matcha Tea are Green – both rely on Green Tea Plants while Matcha Tea “is finely ground powder of specially grown and processed green tea leaves” (Wikipedia). Black Tea has a Brown Color, mixed with Milk it gets a light brown or Beige tone – as Coffee with Milk. Green Tea and Black Tea are translucent, while Matcha Tea, Black Tea with Milk and Coffee with Milk are not transparent and milky.

All of our into the Machine Learning Process fed pictures are out of Image Search Engines. Most of the pictures are for some kind of advertising purpose on the internet, which plays a role in our data. Further objects on the pictures as the containers of the teas (Glasses, Cups, etc.) and the all around stuff on the pictures (blossoms, powder, decoration objects, etc.) are often on the pictures. We tried to not feed misleading stuff into the ML-Process outside of the container; we tried to avoid that some green tea pictures as well as matcha pictures had identical decorational objects for example. A second thing the pictures visually are influenced by is some Image editing, for example the green tea is often far more shiny green than it is in reality.

Analyzing the Data before feeding it into the machine it was quite easy to recognize the difference from Black Tea to both Green Teas, just because of the color. The distinction between Green Tea and Matcha is quite easy too, because of the distinctive milky and fluffy structure. For us there was not much distinction between Black Tea with Milk and Coffee with Milk inside the Data. If you want so, Black Tea with Milk is sometimes a bit lighter and more red than Coffee with Milk, which is often a bit darker.

Testing Scenarios

In the recognition tests we made two test runs: one test with images out of search engines, one test with the webcam and self brewed tea and coffee.

Webcam Test

We unfortunately had no matcha tea, so we used additionally some very green cleaning liquid to try to address the Green Tea-Class:

Left: Fake Green Tea (Cleaning Liquid), Upper Row Left: Coffee with Milk, Upper Row Right: Black Tea, Lower Row Left: Black Tea with Milk, Lower Row Right: Real Green Tea

Black Tea: recognized perfectly
Green Tea: recognized wrongly as Coffee with Milk
Black Tea with Milk: recognized wrongly (depending on exposure) either as Green Tea, Coffee with Milk or Matcha Tea
Fake Green Tea: “correctly” recognized as Green Tea
Coffee with Milk: mostly recognized as Matcha Tea

It seems like the webcam situation bring up some problems like the Black Tea with Milk is depended on how the position of the glass is; because of that the results are that different. Thats why we made the following tests.

Test with Pictures out of Search Engines

In this testing scenario we took pictures out of search engines to try how the machine processed our input.

In the shown test-scenario we tested, if the machine learning process could recognize the difference between Tee with Milk and Coffee with Milk. We payed attention that we not use any pictures with too much foam (which is usually a sign for Coffee – even in the Dataset). The Teachable Machine recognized most of the images right, we think that that is mostly an output from the tendency of Tea with Milk to be red and or either be more lighter. Only the middle picture of the Lower Row was recognized wrongly, its a tea and recognized as a coffee (78% coffee with milk, 22% Tea with Milk).
(Upper Row: Tea, Tea, Coffee; Lower Row: Tea, Tea, Coffee)

Apart from that the machine could really reliable distinguish between Matcha an Green Tea and the other ones. It even had mostly no problem to distinguish between a real Green Tea, a "photoshopped" Green Tea and a black tea as shown in the pictures above.

Test the Machine Learning Process with flat colors

To just test one layer of our data, we tried the Machine Learning Process with images just filled with flat colors to look up, if our assumption that more "red" inside of the colors bring the ML-Process to think that its always Tee with Milk and not Coffee with Milk.

Our Assumption with "red", as the tests above shows, could be mostly wrong.

All Color Tests tend to be recognized as Coffee with Milk over all.


The Machine learned better than we thaught to distinguish between the Tea Types and the Coffee with Milk. All of the easy assumptions we tried to bring up, could not be fully validated by the tests. In the Webcam Tests the results seem to be influenced by the webcam situation in front of the cam. Reflections in the cam and background seems to bring up a lot of false test-interpretations. The recognization of other images is very well done by the machine. All assumptions made up before about how color is related to the classes could be falsified in the end by a test with images just in one color.

Test our ML Process

You can try the Machine Learning Process by dragging a picture into the black square:

Or use your Webcam and present a Drink in front of your Device: