wiki:Developer/FeatureService/Api

Feature Service API - HTTP

Documentation

The documentation portion of the API is meant to provide the user with information to guide them on how to use the feature service HTTP interface and options the feature service provides (feature names and info, etc.). This section will show users how to easily find this information and how one can use it.

Below shows a response from a request to the root path of the feature service.

GET  http://host/features
==>
    <resource uri="http://host/features">
        <command name="FEATURE_NAME" value="Documentation of specific feature"/>
        <command name="list" value="List of available features"/>
        <command name="formats" value="List of output formats"/>
        <command name="format/FORMAT_NAME" value="Documentation of specific format"/>
        <command name="/FEATURE_NAME/FORMAT_NAME?image|mask|gobject=URL[&image|mask|gobject=URL]" value="Returns feature in format specified"/>
   </resource>

Displayed is a list of commands in XML of the API's accepted commands. The attribute name shows the paths from root and the value is a little description of the command.

A very useful piece of information to know about the feature service is what type of features can be calculated. The command below will provide the list in XML.

GET  http://host/features/list
==>
    <resource uri="http://host/features/list" description="List of available features">
        <library name="OpenCV">
            <feature name="SURF" permission="Published" uri="/features/SURF"/>
            <feature name="SIFT" permission="Published" uri="/features/SIFT"/>
            <feature name="ORB" permission="Published" uri="/features/ORB"/>
            ...
        </library>
        <library name="MPEG7Flex">
            <feature name="CLD" permission="Published" uri="/features/CLD"/>
            <feature name="CSD" permission="Published" uri="/features/CSD"/>
            ...
        </library>
        ...
    </resource>

The features are broken up into libraries. Each library normally refers to the code base that is used to calculate a particular feature. The feature names are the names to request the feature service for that specific feature. Most of the time they will be an abbreviation of the actual feature name for everyone's convenience. If the user wanted more information about the feature like the full name, a description of the feature, how long the feature is or the required input resources. The following command will provide that information.

GET  http://host/features/*feature name*

Example...

GET  http://host/features/CLD
==>
    <resource uri="http://host/features/CLD">
        <feature name="CLD">
            <tag name="cache" value="True"/>
            <tag name="confidence" value="good"/>
            <tag name="description" value="Color Layout Descriptor"/>
            <tag name="feature_length" value="120"/>
            <tag name="required_resource" value="image"/>
            <tag name="additional_resource" value="mask,gobject"/>
        </feature>
    </resource>

Show above is the info given on CLD. Cache shows whether the feature is cached in the feature services HDF5 feature tables. Confidence is how reliable the extraction algorithm is currently. Description and feature length are self explanatory. Required resource is the type of resource required for every calculation of this feature while additional resource is the resources that can be request with the required resource to provide more customization for a feature calculation.

Another nice piece if information to know is the types of outputs the user can expect from the feature service. To view the list, just provide the command below.

GET http://host/features/formats
==>
    <resource uri="http://host/features/formats" description="List of Return Formats">
        <format name="xml" permission="Published" uri="features/format/xml"/>
        <format name="hdf" permission="Published" uri="features/format/hdf"/>
        <format name="csv" permission="Published" uri="features/format/csv"/>
    </resource>

The list of response formats are returned

More information about each response format can be requested with this command.

GET http://host/features/formats/**format name**

Example...

GET http://host/features/formats/xml
==>
    <resource uri="http://host/features/formats/xml">
        <format name="XML">
            <tag name="Name" value="XML"/>
            <tag name="content_type" value="text/xml"/>
            <tag name="Description" value="Extensible Markup Language"/>
        </format>
    </resource>

Feature Service Operations

Now that we got the info section of the feature service API out of the way. Here are the commands for the heavy lifting operation and a few example of their responses.

Generally, an individual feature request will be composed of three parts: the feature that is being request, the response format the user wanted to be returned, and a query section which is composed of the resource or resources the user want the feature to be calculated on. In some cases, the query portion will also contain a query argument. The query section will be explained in more detail in the feature query section. Below is a simple get request to the feature service and an example.

GET  http://host/features/*feature name*/*format*?*feature_query*

Example...

GET http://host/features/CLD/xml?image=http://host/image_service/image/00-wqmptTLNAaYXiguu95gtUk
==>
    <feature type="CLD" uri="http://host/features/CLD/xml?image=http://host/image_service/image/00-wqmptTLNAaYXiguu95gtUk" value="50,15,26,27,17,13,17,18,...,16,16"/>

This request asked the feature service for the feature CLD (Color Layout Descriptor) with a response as XML on the image resource located at "http://host/image_service/image/00-wqmptTLNAaYXiguu95gtUk". The response was return in XML as requested. The resource URI is provided with the full request. A feature resource is show with the attributes as the feature and the resources. Tag resources are children of the feature resources with the feature vector displayed as a value of the tag named feature. Other tags can also be provided as further information of the feature but a feature resource will always have a feature tag with value as the feature vector.

A GET request will only get the user one feature. One feature maybe be all the user needs but in most cases one needs many features. To request for a set of features, a POST of a list of features in XML must be made to the feature service. The request is provided below along with an example.

POST  http://host/features/*feature name*/*format*
    <resource>
        <feature type="**feature name**" uri="http://host/features/*feature name*/*format*?image|mask|gobject=URL[&image|mask|gobject=URL]"/>
        ...
    </resource>

Example...

POST  http://host/features/CLD/xml
    <resource>
        <feature type="CLD" uri="http://host/features/CLD/xml?image=http://host/image_service/image/00-wqmptTLNAaYXiguu95gtUk"/>
        <feature type="CLD" uri="http://host/features/CLD/xml?image=http://host/image_service/image/00-xVjcYevFhie6KSpUFAX8HFa"/>
        ...
        <feature type="CLD" uri="http://host/features/CLD/xml?image=http://host/image_service/image/00-nZGu2wS3yVqfLFij4jjDmN"/>
    </resource>
==>
    <resource uri="http://host/features/CLD/xml">
        <feature type="CLD" uri="http://host/features/CLD/xml?image=http://host/image_service/image/00-wqmptTLNAaYXiguu95gtUk" value="50,15,26,27,17,13,17,18,...,16,16"/>
        <feature type="CLD" uri="http://host/features/CLD/xml?image=http://host/image_service/image/00-xVjcYevFhie6KSpUFAX8HFa" value="30,20,75,12,10,12,47,28,...,23,34"/>
        ...
        <feature type="CLD" uri="http://host/features/CLD/xml?image=http://host/image_service/image/00-nZGu2wS3yVqfLFij4jjDmN" value="77,23,50,10,5,23,32,19,...,16,16"/>
    </resource>

The first two parts are the same as a GET request with the feature to be request on and the format for the feature service to return with are the same. There is no query since that information will be contained in the XML document posted along with this request. The XML document open with a resource tag. Its children will be feature tags with the attributes as the resources and the query arguments. Each new feature node of the resource tag will be another feature provided to the feature service. The type attribute is not required and is ignored when parsing the document. All the features will be calculated with the feature given in the request URL. The response this time is in XML and it is similar as the GET response shown earlier except more features will be returned.

Formats

The feature service can return features in a few different formats. The current formats support are shown here. More may be added at a later time however for large sets of features the HDF5 format is preferred.

The response format type is always specified in the URL request. Here are some example of each format:

XML

GET http://host/HTD/xml?image=http://host/image_service/image/aC89AMvPi4QQZdyYa29Fmj
==>
    <feature type="HTD" uri="http://host/HTD/xml?image=http://host/image_service/image/aC89AMvPi4QQZdyYa29Fmj" value="2.9597,1.49856,1.50419,2.96377,1.50131,1.49637,...,21.2668,53.3995,21.0907,20.1079"/>

This is a topical response for a feature in XML. Here is a less typical response.

GET http://host/features/HTD/xml?image=http://host/image_service/image/00-xkvTVnuMiHSvGop6HYeFni&mask=http://host/image_service/image/00-phbr7YqrXxpWQzG3kaTuHn
==>
    <resource uri="http://host/features/HTD/xml?image=http://host/image_service/image/00-xkvTVnuMiHSvGop6HYeFni&mask=http://host/image_service/image/00-phbr7YqrXxpWQzG3kaTuHn">
        <feature type="HTD" "http://host/features/HTD/xml?image=http://host/image_service/image/00-xkvTVnuMiHSvGop6HYeFni&mask=http://host/image_service/image/00-phbr7YqrXxpWQzG3kaTuHn" value="0.277983,0.13343,0.121016,0.224688,0.122258,0.136079,...,13.5971,9.12559,10.7777,12.5585,20.6052"/>
            <tag name="region" value="0" type="number"/>
        </feature>
        <feature type="HTD" image="http://host/image_service/image/00-xkvTVnuMiHSvGop6HYeFni&mask="http://host/image_service/image/00-phbr7YqrXxpWQzG3kaTuHn" value="0.390138,0.205496,0.189508,0.366211,0.195351,0.206468,...,21.4478,14.8232,25.2616,27.7752,22.519"/>
            <tag name="region" value="1" type="number"/>
        </feature>
        ...
    </resource>

If only one feature is returned; the root tag will be feature. When more feature vectors are returned the list of feature nodes are enclosed in a resource node as the root. Attributes of the feature node are query arguments that produced the feature vector and a type attribute displays the feature name. As presented earlier in the operation section, tags nodes are children of the feature node. If only a feature vector is returned with no extra information no tags will be present, however some features will provide extra information about the specific feature like the scale or location the feature was attracted from in the images. This information will be presented in the tag nodes.

Example...

    <resource uri="http://host/features/SIFT/xml?image=http://host/image_service/image/00-JSrYb7cH8Y2EX28DX9PoF3">
        <feature uri=?"http://host/features/SIFT/xml?image=host/image_service/image/00-JSrYb7cH8Y2EX28DX9PoF3" type="SIFT" value="3,2,1,0,0,0,2,3,38,2,1,4,1,1,4,20,189,0,0,0,0,0,...,5,2,2,0,1,1,29">
            <tag name="x" value="127.943" type="number"/>
            <tag name="y" value="27.4335" type="number"/>
            <tag name="response" value="0.0171659" type="number"/>
            <tag name="size" value="2.18904" type="number"/>
            <tag name="angle" value="112.98" type="number"/>
            <tag name="octave" value="1.42874e+07" type="number"/>
        </feature>
        ...
   </resource>
Error Handling

Not every time a feature will be calculated correctly. Instead of returning an error for the entire response the feature service returns a partial response with error responses on the fail calculated feature vectors. An error response for XML is shown below. The entire document will be presented with a 200.

GET: http://host/features/HTD/xml?image=http://host/image_service/image/not_valid_id
==>
    <feature type="HTD" uri="http://host/features/HTD/xml?image=http://host/image_service/image/not_valid_id">
        <tag name="error" value="403:User is not authorized to read resource internally"/>
    </feature>

No feature vector will be show since it could not calculate the vector but an error code and a message will be presented in the name/value format as tags of features.

CSV

Here is a few examples of response for CSV:

GET http://host/HTD/csv?image="http://host/image_service/image/aC89AMvPi4QQZdyYa29F
==>
    index,feature type,image,descriptor,error
    0,HTD,http://host/image_service/image/00-xkvTVnuMiHSvGop6HYeFni,"0.664822,0.345364,0.330927,...12.0382,44.5796,36.1573,29.2452,25.5638,27.7235,32.2795"

Again here is a less typical example.

GET http://host/features/HTD/csv?image=http://host/image_service/image/00-xkvTVnuMiHSvGop6HYeFni&mask=http://host/image_service/image/00-phbr7YqrXxpWQzG3kaTuHn"
==>
    index,feature type,image,mask,region,descriptor,region,error
    0,HTD,http://host/image_service/image/00-xkvTVnuMiHSvGop6HYeFni,http://host/image_service/image/00-phbr7YqrXxpWQzG3kaTuHn,"0","0.277983,0.13343,0.121016,...,5.97154,39.7101,13.5971,9.12559,10.7777,12.5585,20.6052",0,
    1,HTD,http://host/image_service/image/00-xkvTVnuMiHSvGop6HYeFni,http://host/image_service/image/00-phbr7YqrXxpWQzG3kaTuHn,"1","0.390138,0.205496,0.189508,...,7.3215,45.6364,21.4478,14.8232,25.2616,27.7752,22.519",1,
    2,HTD,http://host/image_service/image/00-xkvTVnuMiHSvGop6HYeFni,http://host/image_service/image/00-phbr7YqrXxpWQzG3kaTuHn,"2","0.435565,0.207495,0.183849,...,8.11679,46.853,21.387,14.062,23.3983,27.3495,23.8318",2,
    3,HTD,http://host/image_service/image/00-xkvTVnuMiHSvGop6HYeFni,http://host/image_service/image/00-phbr7YqrXxpWQzG3kaTuHn,"3","0.41636,0.214488,0.205545,...,8.91934,34.3196,10.3958,7.47228,12.2179,27.927,24.1924",3,
    4,HTD,http://host/image_service/image/00-xkvTVnuMiHSvGop6HYeFni,http://host/image_service/image/00-phbr7YqrXxpWQzG3kaTuHn,"4","0.376811,0.203457,0.202662,...,5.56578,29.2855,26.5095,16.3125,22.5986,25.9788,20.3697",4,
    ...

The first column is the index number all the feature vectors returned followed by the feature name and then the resource type used to calculate each feature vector. The resource types can vary depending on the feature requested. The descriptor column is the column with the feature vector. The format will always be "*number*,*number*,...,*number*". Any parameters will be in the subsequent columns and can be either strings or numbers values and depends completely on the feature used. There is an extra column error reserved for if an error occurs in the feature extraction.

Error Handling

Error handling for CSV is pretty simple. If an error occurs there will be a column added as the last column labeled error. In the case of an error during calculation of a feature or an authentication error or a resource the error code and message will be displayed accordingly. Like the XML response format the entire document will be returned with a 200.

Example...

GET http://host/features/HTD/csv?image=http://host/image_service/image/not_valid_id
==>
    index,feature type,image,feature,error
    0,HTD,http://host/image_service/image/not_valid_id,,0,403:Resource not found

HDF5

The HDF5 format is typically the best format to use if trying to extract features from a large data set. To interface with the hdf5 we currently use Pytables (http://www.pytables.org/moin) and to view we use a java app(http://www.hdfgroup.org/products/java/index.html). The java app is for debugging and would not recommend it for very large HDF5 files. A quick example below shows how a hdf5 format is requested and returned by the features service.

GET http://host/HTD/hdf?image=http://host/image_service/image/aC89AMvPi4QQZdyYa29F
==>

As the java app viewer shows, there is a columns for the feature and all the resources to create the features. If there are any parameters that are returned with the feature, they will also be shown as new columns. Each new row will be for a new feature.

Error Handling

A table of response codes is also included in the HDF5 file. This tables rows correspond to the feature tables rows. Most of the time that table will be filled with 200 but if the user provided the feature service with a resource that the user has no authorization to calculate on a 403 will be returned for that resource in the response code table. The entire response will receive a 200 since no error occurred in the feature service construction of the response.

Feature Query

The query section of a feature service request provides the user with a lot of versatility. This section of the request to the feature server is composed of all the relevant resources a feature extraction requires to produce a result.

Query Arguments

Currently, there are only three query arguments accepted by the feature service: image, gobject, and mask

  • image

An image service URL should be given to the image argument. It does except none image service URLs that point to images however some of the image conversions and pixel operations are done on the image service so your image has to be of the correct format accepted by that feature or one can make there life a lot easier by just uploading the image to bisque and pass the image service URL instead. Another nice gain by using the image service are that the user can pass image service URL with image operations on the argument. Here is a quick example:

http://host/features/HTD/xml?image=http://host/image_service/image/00-JZC9xshEif6CdF6SukSCU5?transform=edge

From this example the image service edge transform is applied to the image before the feature calculation is performed providing a user with a lot of customization in the URL request.

  • gobject

The gobject argument accepts a gobject url. If gobjects are a new concept please look at http://biodev.ece.ucsb.edu/projects/bisquik/wiki/Developer/Metadata/Graphical to get up to speed on gobject. The way a gobject is used in the feature service depends greatly on the feature that is being requested. Here is an example to use the gobject as a mask to get the region of interest.

http://host/features/HTD/xml?image=http://host/image_service/00-JZC9xshEif6CdF6SukSCU5&gobject=http://host/data_service/00-JZC9xshEif6CdF6SukSCU5/gobject/2638911

This polygon gobject region that it outlines can be taken as a region of interest for the feature. This future will only accept curtain gobject that are shape. It will not know what to do with a point gobject and will return an error.

  • mask

The mask argument accepts image service URL like the image argument however the mask image is applied to the given image as a mask. This mean each group of pixel values are applied as a feature calculation. This allows segmentation to be used to break images for more localized feature calculations. Here is an example using super pixels with the image service and feature service to calculate the regions.

http://host/features/CLD/xml?image=http://host/image_service/image/00-wqmptTLNAaYXiguu95gtUk&mask=http://host/image_service/image/00-wqmptTLNAaYXiguu95gtUk?transform=superpixels

This will return CLD features for as many superpixel regions are found.

To know what resource to use with which feature can be a little confusing. This is where the required resource and the addition resource in the documentation can come in handy. While what the addition resource and require resource results can be found in the feature documentation on the wiki the accept queries for a specific feature will be provided in the documentation. The required resource as its name suggest is a resource that is required for every feature calculation for that particular feature. For CLD and image resource is always required whether you provide and mask or a gobject. If you do not provide the required resource and error will result so pay attention to the require source tag. The addition resource are resources you can provide along with the required resource for extra feature calculation functionality. We already explain for some features the gobject and the mask can allow for a user to calculate a region of interest. For key point features a point gobject can be provided to specify a specific key point on the image. Right now these are the only uses of these additional resources. More uses may be coming soon along with more features.

Query Examples

So that was an intro to the way to calculate features on resource. This section will show more intricate and interesting uses of feature service with data service and image service to customize feature extractions.

Image service allows for images to be broken into tiles. This can be useful for feature calculations when one wants small localize region to extract a feature. The image service tile functionality is exploited in the Botanicam module to break each image into tiles to calculated Homogenous Texture Descriptor on. This provides more examples and better localized regions for the support vector machine classifier. Below a request similar to the one the Botanicam module makes.

POST  http://host/features/CLD/hdf
    <resource>
        <feature type="CLD" uri="http://host/features/CLD/xml?image=http://host/image_service/image/00-wqmptTLNAaYXiguu95gtUk?tiles=64,0,0,0"/>
        <feature type="CLD" uri="http://host/features/CLD/xml?image=http://host/image_service/image/00-wqmptTLNAaYXiguu95gtUk?tiles=64,1,0,0"/>
        <feature type="CLD" uri="http://host/features/CLD/xml?image=http://host/image_service/image/00-wqmptTLNAaYXiguu95gtUk?tiles=64,2,0,0"/>
        ...
        <feature type="CLD" <feature type="CLD" uri="http://host/features/CLD/xml?image=http://host/image_service/image/00-wqmptTLNAaYXiguu95gtUk?tiles=64,100,100,0"/>/>
    </resource>

Since we need features on many tiles we make a POST of a list of resources. Each resource calls to the image service to return a different tiles on the image. So each resource may take a bit of time to calculate on. To improve the time, the Botanicam module takes advantage of another property of the bisque system. The requests can be made in paralleled. So groups of features can be request at once to improve performance of a module. Currently the python api will paralize requests with a simple function call. Look into the Python Api documentation for more information. So that is cool! Right? The feature service automatically can parallelize feature request for the user without the user having to worry about it.

Alright so the image service has many more operations than just breaking an image into tiles. There are a few transform like, fourier, edge, radon, chebyshev and more... Also the image can be scaled, sliced and thresholded as the user pleases. This also carries down to the feature service when calculating features on image. Now here is probably a less practical request but shows the power of the image service with the feature service.

Image service request

http://host/image_service/image/00-wqmptTLNAaYXiguu95gtUk?roi=20,20,100,100&remap=grey&transform=chebyshev&transform=edge

Feature service request

http://host/features/HTD/xml?image=http%3A%2F%2Fhost%2Fimage_service%2Fimage%2F00-wqmptTLNAaYXiguu95gtUk%3Froi%3D20%2C20%2C100%2C100%26remap%3Dgrey%26transform%3Dchebyshev%26transform%3Dedge

So that is a very complicated image service request and is probably useless but shows what the image service combine with the feature service are capable of accomplishing. Also not that the image service request is encoded this time. If the image service include & signs in the request it has to be encoded to the request does not get included with the feature service query argument. This may confuse the feature service on what operation to perform.

Alright. So we did not really talk to much about the key point features like SIFT, SURF, ORB, and BRISK. There may be more to come and this fact will apply to those as well. These key point features can except gobject however they do not work with all gobject, just point gobject. So an image can be provided by itself and a set of key point will be extracted and return with there corresponding features. Now if you want to apply keypoint of your own you can use the point gobject to pick a key point. All the user needs to do is create a point gobject or a set and post them to the data service on the image, then request the image and the gobject for these features resulting in a feature calculated at that keypoint. Here is an example of a request.

http://host/features/SIFT/xml?image=http://host/image_service/image/00-wqmptTLNAaYXiguu95gtUk&gobject=http://host/image_service/image/00-wqmptTLNAaYXiguu95gtUk/gobject/1234642

This will provide one keypoint feature at that point. If you provide other gobject this request will fail, so remember the type of gobject also matter for a particular feature calculation.

So now that we have gone over intersting ways to use images and gobject, lets go over a cool way to use masks with the image service. So the image service also provides a function to calculate superpixels on an image. We can use those superpixels as masks to localize feature calculations on each superpixel. Here is a request example. The image service argument and the mask argument are shown separately to show the request before it is encoded.

Image Argument:

Image Service URL: http://host/image_service/image/00-wqmptTLNAaYXiguu95gtUk?roi=20,20,100,100&remap=grey&transform=chebyshev

Mask Argument:

Image Service URL: http://host/image_service/image/00-wqmptTLNAaYXiguu95gtUk?roi=20,20,100,100&transform=superpixels

to the feature service.

GET http://host/features/CLD/xml?image=http://host/image_service/image/00-wqmptTLNAaYXiguu95gtUk%3Froi%3D20%2C20%2C100%2C100%26remap%3Dgrey%26transform%3Dchebyshev&mask=00-wqmptTLNAaYXiguu95gtUk%3Froi%3D20%2C20%2C100%2C100%26transform%3Dsuperpixels

In this example a region of the image is picked out, converted to gray scale and a chebyshev transform is applied. On the masked sided the region is picked out and superpixels are found on the colored image. This example shows the versatility of the feature service along with the versatility of other bisque services.

Last modified 3 years ago Last modified on Nov 21, 2014 12:10:26 AM

Attachments (1)

Download all attachments as: .zip