wiki:Developer/FeatureService/ApiPython

Feature Service API - Python

BQApi provides bisque users with a means to extract features from resources using the feature service. Below is information on the python API and a few example on how to use it.

bqfeatures

FeatureResource(image=None, mask=None, gobject=None)

Named tuple to make it easier to organize resources. Of course, one can always just make a simple list of tuples for the resource list.

class Feature()

The Feature class is the base implementation of the feature service API for when one need to extract a set of features on a small set of resources. BQSession is used as the communication layer for bqfeatures, therefore before making any requests a local BQSession has to be instantiated to pass to any feature request.

fetch(session, name, resource_list, path=None)

Requests the feature server to calculate features on provided resources.

  • session - a local instantiated BQSession attached to a MEX.
  • name - the name of the feature one wishes to extract.
  • resource_list - list of the resources to extract. Format: [(image_url, mask_url, gobject_url),...] if a parameter is not required just provided None.
  • path - the location were the HDF5 file is stored. If None is set, the file will be placed in a temporary file and a Pytables file handle will be returned. (default: None)
  • return - returns either a Pytables file handle or the file name when the path is provided

Lets upload an image an calculate a single feature on it.

import os
from bqapi.comm import BQSession
from bqapi.bqfeature import Feature, FeatureResource

session = BQSession().init_local(user, pwd, bisque_root='http://bisque.ece.ucsb.edu') #initialize local session

response_xml = session.postblob('myimage.jpg') #post image to bisque and get the response

#construct resource list of the image just uploaded
resource_list = [FeatureResource(image='http://bisque.ece.ucsb.edu/image_service/%s' % response_xml.attrib['resource_uniq'])]

#fetch features on image resource
pytables_response = Feature().fetch(session, 'HTD', resource_list)

#get a numpy list of features for the downloaded HDF5 file.
feature = pytables_response.root.values[:]['feature']

#close and remove the HDF5 file since it is stored in a tempfile
pytables_response.close()
os.remove(pytables_response.filename)
fetch_vector(session, name, resource_list)

Requests the feature server to calculate features on provided resources. Designed more for requests of very few features since all the features will be loaded into memory.

  • session - a local instantiated BQSession attached to a MEX.
  • name - the name of the feature one wishes to extract
  • resource_list - list of the resources to extract. format: [(image_url, mask_url, gobject_url),...] if a parameter is not required just provided None
  • return - a list of features as numpy array
  • FeatureCommError (exception) - if any part of the request has an error the FeatureCommError will be raised on the first error. (Note: You can use fetch and read from the status table for the error. Warning: fetch_vector will not return response if an error occurs within the request)

Fetch vector is the quick and easy way to just get the feature vector back as a Numpy array. Note the feature service may rearrange the order of the resource list so the rows of the returned Numpy matrix do not necessarily correspond to the element of the resource list.

length(session, name)

Static method that returns the length of the feature requested

  • session - a local instantiated BQSession attached to a MEX.
  • name - the name of the feature one wishes to extract
  • return feature length

class ParallelFeature(Feature)

For faster and larger calculations, features can be requested in parallel. ParallelFeature provides a parallel version of the Feature class. Not much changes as far as the API is concerned; but, speed ups will be seen in the time it takes to calculate a large set of features. The resource list is divided depending on its size and the max thread and chunk size set as default for the ParallelFeature. Each of the smaller requests are stored in temporary files to be merged with a large request file. The user can override these parameters with the functions below as they see necessary. Depending on the size and speed of the bisque server other sizes may be more optimal than the default. Note that both have to be set for the max thread and chunk size to be over written.

set_thread_num(n)

Overrides the internal thread parameters, chunk size must also be set to override the request parameters

  • n - the number of requests made at once
set_chunk_size(n)

Overrides the chunk size, thread_num must also be set to override the request parameters

  • n - the size of each request
fetch(session, name, resource_list, path=None)

Requests the feature server to calculate provided resources. The request will be boken up according to the chunk size and made in parallel depending on the amount of threads.

  • session - the local session
  • name - the name of the feature one wishes to extract
  • resource_list - list of the resources to extract. format: [(image_url, mask_url, gobject_url),...] if a parameter is not required just provided None
  • return - returns either a Pytables file handle or the file name when the path is provided

Here we create a list of image resources from a dataset and extract HTD on each image.

import os
from bqapi.comm import BQSession
from bqapi.bqfeature import Feature, FeatureResource
import tables

session = BQSession().init_local(user, pwd, bisque_root='http://bisque.ece.ucsb.edu') #initialize local session

response_xml = session.fetchxml('http://bisque.ece.ucsb.edu/data_service/*dataset id*/values') #fetch dataset elements

#construct resource list of image service requests on dataset elements
resource_list = []
for image in response_xml.xpath('image'): #using xpath to find all the image elements
    resource_list.append(FeatureResource(image='http://bisque.ece.ucsb.edu/image_service/%s' % image.attrib['resource_uniq'])) 

#fetch features for set of resources
hdf5_filename = ParallelFeature().fetch(session,'HTD',resource_list, path='myfeatures.h5')

with tables.open_file(hdf5_filename,'r') as h5file:
   feature_list = h5file.root.values[:]['feature']

This example is very similar to the none parallel features. Instead of storing a tempfile a path was given to store the file locally so it can be retrieved later.

fetch_vector(session, name, resource_list)

Requests the feature server to calculate provided resources. The request will be broken up according to the chunk size and made in parallel depending on the amount of threads.

  • session - the local session
  • name - the name of the feature one wishes to extract
  • resource_list - list of the resources to extract. format: [(image_url, mask_url, gobject_url),...] if a parameter is not required just provided None
  • return - a list of features as numpy array

The requests may not all go smoothly. If an error occurs this callback function is called. Currently it just returns a warning log that the response returned an error. A user can replace this callback function with whatever they want to do when an error occurs. Maybe store all the errors in a list or store the last or first error. This function gives some amount of versatility when an error occurs in ParallelFeature requests.

errorcb(e)

Called when an error occurs. Meant to be overwritten for flexibility in the event of a comm error

  • e - a bqcommerror exception

Source references

Last modified 3 years ago Last modified on Nov 24, 2014 4:23:32 PM