[Feature] convert mask image into json

[slchenchn]

Hello, I was wondering if it possible to convert an annotated image generated by my network into labelme's json file format, given the original image and the mask image available? I think it may be possible, but I don't know how to code it.

[GuessWhosBack]

Hello, I have been wondering the same thing.

[qwert31639]

Maybe this code can give you some idea

import contextlib
from labelme import PY2
import numpy as np
import json
import io
from skimage.io import imread,imshow,imsave
import skimage.morphology as sm
import cv2
ENCODING = 'utf-8'

IMAGE_NAME = 'path/to/your/mask/'
JSON_NAME = 'path/to/your/jason/path'

@contextlib.contextmanager
def open(name, mode):
assert mode in ["r", "w"]
if PY2:
mode += "b"
encoding = None
else:
encoding = "utf-8"
yield io.open(name, mode, encoding=encoding)
return

all_point = []
image = 255 - (imread(IMAGE_NAME)[:,:])
image_bwlabel = np.where(image>0,255,0)
label_image = sm.label(image_bwlabel)
point_number = 0
for number in range(1,label_image.max()):
point = np.zeros(image.shape)
point[np.where(label_image == number)] = 255
contours, hierarchy = cv2.findContours(point.astype(np.uint8),cv2.RETR_TREE,cv2.CHAIN_APPROX_TC89_L1)

for c_number in contours:
    if c_number.shape[0] > 5:
        point_list = c_number[:,0,:].astype(np.float)
        if point_list.shape[0] > 20:
            point_list = point_list[0::2]
        if point_list.shape[0] > 20:
            point_list = point_list[0::2]

  
        print(point_list.shape)
        point_number = point_number + point_list.shape[0]
        point_list = point_list.tolist()    
        inner_dict = {"label" : 'a',
                      "points" : point_list,
                      "group_id": None,
                      "shape_type": "polygon",
                      "flags" : {},
                      }
        all_point.append(inner_dict)

out_dict = {'version':"4.5.6",
"flags": {},
"shapes" : all_point,
"imagePath": IMAGE_NAME.split('/')[-1].split('.')[0] + '.bmp',
"imageData": None,
"imageHeight" :image.shape[0],
"imageWidth" :image.shape[1],
}

with open(JSON_NAME, "w") as f:
json.dump(out_dict, f, ensure_ascii=False, indent=2)

[Sparknzz]

Using opencv findContours seems a simple way, but it leads to dense points issue (eg. a 800*800 image, will generate 3000 points and hard to drag or move them in qt).

[KristynaPijackova]

Hi, I was wondering this question as well and found quite a simple solution on stack overflow with a help of a library called imantics.

from imantics import Polygons, Mask

img_path = 'path_to_your_img' 
image = cv2.imread(img_path)
outputs = predict(image)

pred_img_mask = outputs['instances'].pred_masks.numpy()*1
pred_poly_points = []

for i in range(pred_img_mask.shape[0]):
    polygons = Mask(pred_img_mask[i]).polygons()
    pred_poly_points.append((polygons.points)[0])

This way you get a polygon array with the same formatting as in the labelme files, for the same format as in COCO like datasets you can use polygons.segmentation instead.

Edit: Oh, maybe I should add, that I used the detectron2's segmentation model (R50 FPN Mask R-CNN) to get ou the prediction and the mask, so that part of the code may differ for other codes, but the output I get is boolean tensor array (thus I convert it to numpy() and multiply by one) that gets me array of 0s and 1s which has following proportions [# of predicted segmentation' masks, width, height]

[randomrushgirl]

what if i want to include the image data?
This is what I tried but it was not matching and the image was not generated.
`

import base64
from labelme import LabelFile
...
imgBytes = LabelFile.load_image_file(img_path)
data['imageData'] = base64.b64encode(imgBytes).decode('utf-8')

`

[KristynaPijackova]

@randomrushgirl I think I had a similar issue as you do at first... if I remember correctly I think I ended up going through my new generated json file trying to figure out the issue by comparing it to a working one to figure out what might be wrong...

If you think the generated img_data is the problem, what I think should work is generating it, copying it into some working json file (replace the original image data with your new one) and if you see your original labels on a new img background in labelme, then you know the base64 is not the issue, but the json formatting is ( I think that was my case and it was something really stupid like ' ' instead of " " which made it not work)

However I generate the data like this and it works alright:

data = labelme.LabelFile.load_image_file(img_write)
img_data = base64.b64encode(data).decode('utf-8')

I pass the img_data into my outer dict which looks like this:
(note it's not the whole code - I also have a inner dict which I loop through at first)

 out_dict = {"version":"3.15.2",
                    "flags": {},
                    "shapes" : all_points,
                    "lineColor": [0, 255, 0, 128], 
                    "fillColor": [255, 0, 0, 128],
                    "imagePath": "../" + img_name + "." + str(i_img) + ".png",
                    "imageData": img_data,
                    "imageHeight" : img_size,
                    "imageWidth" : img_size,
                 }

[randomrushgirl]

@KristynaPijackova you're right on the money, the image is different but the labelled annotations is as expected.

I generate my data and pass it into an outer dictionary in a similar way you do so I checked my code after I saw your reply and the issue seemed to lie with how I was passing my image data to the JSON file. Thanks for the help! 😄