Computer Vision Coding Example: Objective-C Metal

CMSR ML Studio generated program source codes are very easy to integrate into your applocations. This page will show how to code to use Computer Vision functions. Depending on modeling types, calling parameters are different.

Model Code Generation and Compilation

From Xcode, create a new metal file "CNNForward.metal" and copy the content of the downloaded "CNNForward.metal" file. And Create "MetalObjcModel.h" and copy the content of the downloaded "MetalObjcModel.m" file.

From CMSR ML Studio, perform the followings to generate model codes;

1. Open your computer vision model.
2. From "Modeling" menu, select "Export model as program code".
3. Select "Language" as "GPU: ObjC/OpenCL - Header" and press the "Copy" button. This will generate Objective-C header file to your system clipboard. Create a model header file into your project folder and copy clipboard content and save, say, "CMSRModel.h".
4. Again, select "Language" as "GPU: ObjC/OpenCL - Main" and press the "Copy" button. This will generate Objective-C body/main file to your system clipboard. Create a main model file into your project folder and copy clipboard content and save, say, "CMSRModel.m".

Copy model parameter file into locations where your main program can access. And modify access paths in your main program. To create model parameter file, from "Modeling" menu, select "Export model as file (MyDataSay, BI Server, CNN, ...etc.)".

CNN/FCN Classification Example

The following code shows how CNN/FCN classification is called. Note that CMSRModel is the model class that CMSR Studio generates.

@import MetalKit;
#import <Foundation/Foundation.h>
#import "CMSRModel.h"

int main(int argc, const char * argv[]) {
    @autoreleasepool {
 
        CMSRModel *model = [[CMSRModel alloc] init];

        char *imagefilepath = "/Users/cho/cnnimages.rgb";
        char *modelfilepath = "/Users/cho/modelfile.cnn";

        int height = 64;
        int width  = 64;
        int IMAGE[height*width*3];
 
        int blackandwhite = 0;
        int r0g1b2 = 1;
        int i;
 	int outclass;

        int maxOutCount = 4;
        int outClassIndex[maxOutCount+1]; // notice 1 extra.
        float outClassProbability[maxOutCount+1]; // notice 1 extra.
        
	// initialize resources;
        [model initializeGpu];
        [model initializeModel: modelfilepath];
 
	// the followings can be repeated and image can get from camera.
        [model populateImageArray: imagefilepath data: IMAGE length: (height*width*3)];
        outclass =  [model evaluate:
            maxOutCount
            outClassIndices: outClassIndex
            outClassProbabilities: outClassProbability
            blackandwhite: blackandwhite
            r0g1b2: r0g1b2
            IMAGEARRAY: IMAGE
        ];
        for (i = 0; i < maxOutCount; i++) {
            printf(" %d: %d %f \n", i, outClassIndex[i], outClassProbability[i]);
        }

	// release resources;
        [model releaseResources];
	return 0;
}

M-CNN Multi-value Output Regression Example

The following code shows how M-CNN multivalue output regression is called. Note that CMSRModel is the model class that CMSR Studio generates.

@import MetalKit;
#import <Foundation/Foundation.h>
#import "CMSRModel.h"

int main(int argc, const char * argv[]) {
    @autoreleasepool {

        CMSRModel *model = [[CMSRModel alloc] init];

	char *imagefilepath = "/Users/cho/mcnnimages.rgb";
 	char *modelfilepath = "/Users/cho/modelfile.mcnn";

	int height = 70;
	int width  = 70;
	int IMAGE[height*width*3];
	int outclass;

        int blackandwhite = 0;
        int r0g1b2 = 1;
        int i;

 	float outValues[2];
        
 	// initialize resources;
 	[model initializeGpu];
 	[model initializeModel: modelfilepath];

	// the followings can be repeated and image can get from camera.
	[model populateImageArray: imagefilepath data: IMAGE length: (height*width*3)];
  	outclass = [model evaluate:
                 outValues
                 blackandwhite: blackandwhite
                 r0g1b2: r0g1b2
                 IMAGEARRAY: IMAGE
             	];
	for (i = 0; i < 2; i++) {
		printf(" %d: %f \n", i, outValues[i]);
 	}

	// release resources;
        [model releaseResources];
	return 0;
}

OD-CNN Object Detection Example

The following code shows how OD-CNN object detection is called. Note that CMSRModel is the model class that CMSR Studio generates. The X/Y coordinates are the center locations of detected object bounding boxes. Output width and height are relative size where whole size is 1.0.

@import MetalKit;
#import <Foundation/Foundation.h>
#import "CMSRModel.h"

int main(int argc, const char * argv[]) {
    @autoreleasepool {

        CMSRModel *model = [[CMSRModel alloc] init];

        char *imagefilepath = "/Users/cho/odcnnimages.rgb";
        char *modelfilepath = "/Users/cho/modelfile.odcnn";

        int height = 245;
        int width  = 245;
        int IMAGE[height*width*3];
        int outclass;
        int areasizefilter = 1;

        int maxOutCount = 4;
        int outClassIndex[maxOutCount+1]; // notice +1 here.
        float outClassProbability[maxOutCount+1]; // notice +1 here.
        float outX[maxOutCount+1]; // notice +1 here.
        float outY[maxOutCount+1]; // notice +1 here.
        float outWidth[maxOutCount+1]; // notice +1 here.
        float outHeight[maxOutCount+1]; // notice +1 here.
   
        int blackandwhite = 0;
        int r0g1b2 = 1;
        int i;

        // initialize resources;
        [model initializeGpu];
        [model initializeModel: modelfilepath];

	// the following can be repeated and image can get from camera.
	[model populateImageArray: imagefilepath data: IMAGE length: (height*width*3)];
        outclass =  [model evaluate:
 		maxOutCount
		outClassIndex: outClassIndex
		outClassProbability: outClassProbability
		outX: outX
		outY: outY
		outWidth: outWidth
		outHeight: outHeight
		blackandwhite: blackandwhite
		r0g1b2: r0g1b2
		IMAGEARRAY: IMAGE
		areasizefilter: areasizefilter
        	];
        for (i = 0; i < outclass; i++) {
            printf(" %d: %d %f %f %f %f %f\n", i, outClassIndex[i], outClassProbability[i], outX[i], outY[i], outWidth[i], outHeight[i]);
        }

	// release resources;
        [model releaseResources];
	return 0;
}

T-CNN Similarity Regression/Face Recognition Example

The following code shows how T-CNN similarity regression is called. Note that CMSRModel is the model class that CMSR Studio generates.

@import MetalKit;
#import <Foundation/Foundation.h>
#import "CMSRModel.h"

int main(int argc, const char * argv[]) {
    @autoreleasepool {
 
        CMSRModel *model = [[CMSRModel alloc] init];

	char *imagefilepath1 = "/Users/cho/tcnnimages1.rgb";
        char *imagefilepath2 = "/Users/cho/tcnnimages2.rgb";
        char *modelfilepath = "/Users/cho/modelfile.tcnn";

        int height = 70;
        int width  = 70;
        int IMAGE1[height*width*3];
        int IMAGE2[height*width*3];

        int blackandwhite = 0;
        int r0g1b2 = 1;
        int i;

        float outvalue;
   
	// initialize resources;
        [model initializeGpu];
        [model initializeModel: modelfilepath];
 
	// the followings can be repeated and images can get from camera;
        [model populateImageArray: imagefilepath1 data: IMAGE1 length: (height*width*3)];
        [model populateImageArray: imagefilepath2 data: IMAGE2 length: (height*width*3)];
        outvalue =  [model evaluate:
            blackandwhite
            r0g1b2: r0g1b2
            IMAGEARRAY1: IMAGE1
            IMAGEARRAY2: IMAGE2
        ];
        printf("OUT VALUE: %f \n", outvalue);
 
	// release resources;
        [model releaseResources];
	return 0;
}