The Naive Bees Competition was hosted on DrivenData by a bootcamp called Metis which teaches data science — it is based in New York and San Francisco. For more details, please click the links provided. That's the text that is, underlined.

In this competition, your task was to identify a bee as a honey bee(Apis species) or a bumble bee(Bombus species), practice image processing and classification techniques and help researchers seeking to protect pollinators from collapse. I chose this challenge because it relates a lot to the work I'm doing in computer vision in my fast.ai projects especially when it comes to initial image processing of a newly acquired set of images not ready for supervised learning yet.

The competition is over but it resurfaced on Datacamp where it is presented as a real-life project in three jupyter notebooks. I enjoyed going through the notebooks. I wish they could add a requirements.txt file or a conda environment I could use to reproduce the project fully. Before going further I recommend going through either the competition winners notebooks or the actual course.

I'll just breeze through some of the important pieces of code that keep coming up in my workflows :

1. Imports normally done in one cell: I normally forget what I imported and I got this from other people not to throw imports all over the place. It can be really confusing especially if you call a function then you have no idea why it's not working. Didn't I import it? 🤔

# Used to change filepaths
from pathlib import Path

# utility used for drawing static and dynamic graphs
import matplotlib.pyplot as plt

# show the plot in the ipython notebook or jupyter notebook
%matplotlib inline

# utility used to see object in python such as ipywidgets, images en cetera
from IPython.display import display

# data structure manipulation 
import pandas as pd

# n-dimensional array library manipulator e.g matrix manipulation
# you can build a neural network library with this
import numpy as np

# Library used for manipulating images as an example opening, cropping, zooming 
from PIL import Image

# For manipulating files within your working directory
import glob

2. Check out a sample of the images: Bumble bee and Honey bee.

# load and view image
image = Image.open("bee.jpg")

# Get the image size
img_size = img.size

print("The image size is: {}".format(img_size))

Kernel density distribution (KDE)

Really nice summary from Matthew Conlen. It creates a smooth curve | given data. Since the histogram is binned this is biased since it is set by the user. You'll get the shape in terms of skew of the data from it. He says that a silhouette of the true distribution will be formed.

Bumble bee KDE

More yellow due to the background and overlap between the red and green channels.

Honey Bee KDE

Bee image from the tutorial. The width and height of the image. Credit: Peter Bull & Emily Miller DrivenData

I also discovered that you can use it to get the differences between images. As an example, you'll have more of one color represented in different images for example if its comparing between landmasses as an example grassland and a desert: More green for the grassland and an interesting property: More Red and Green no blue. Try making a KDE for that

The blue flower peak is strong on the right.

<aside> 🎯 Try get a bunch of images it could be of something you are interested in and try plot the KDE's extract the color channel values and find their means/medians then use unsupervised learning/cosine similarity to find which images are similar to each other and confirm that by looking at the images.

</aside>

3. Making Pipelines

I also liked that they provided a pipeline of transforming images in your filepath ready for computer vision applications. However, sometimes one might need to go one step ahead to edit images while training the Convolutional Neural Network: Version 2 of fast.ai has really cool functionality that you can tweak even further for example check out the method show_batch and ImageClassifierCleaner. I've edited the code a bit from the project.

# grab all the files with the extension jpg e.g bee_1.jpg
image_paths = glob.glob("*.jpg")

def process_image(path):
		
    # grab the image of the images
    img = Image.open(path)

    # create paths to save files to
    bw_path = "saved_images/bw_{}.jpg".format(path.stem)
    rcz_path = "saved_images/rcz_{}.jpg".format(path.stem)

    print("Creating grayscale version of {} and saving to {}.".format(path, bw_path))
    bw = img.convert("L") # grayscale
    bw.save(bw_path)

    print("Creating rotated, cropped, and zoomed version of {} and saving to {}.".format(path, rcz_path))
    rcz = img.rotate(45).crop().resize((100,100)) # 45 degree rotation and resize to 100 by 100
    rcz.save(rcz_path)

# for loop over image paths
for img_path in image_paths:
    process_image(Path(img_path))