Hair and Geometry Shaders

When working with hair primitives from within a geometry shader using placeholder callback functions, you do need to add something extra. There is an admittedly hard to find special note regarding hair primitives and the placeholder callback. Check page 264 of the 2nd edition of the Programming mental ray Handbook. If you're refering to on-line manual, it should be at the very end of the Geometry Shaders section in the Using and Writing Shaders chapter 3.

From the more detailed description with an example on this site, we make a feather using a geometry shader and hair primitives. In the geometry shader function that references the callback, we see:
...
miTag tag;
miObject *obj = mi_api_object_begin(mi_mem_strdup("geofeather"));
mi_api_object_callback(geofeather_cb, (void *)paras);
obj->visible = ...
...
tag = mi_api_object_end();
mi_geoshader_add_result(result, tag);
obj = (miObject *)mi_scene_edit(tag);
obj->geo.placeholder_list.type = miOBJECT_HAIR;
mi_scene_edit_end(tag);
Those last three lines are the necessary additions to inform that it is hair geometry rather than the normal polygons or NURBS.