Add REST resource for rebuilding the search index

Hi,

The API allows the rebuild of the entire search index, for a given object or for all entities of a given type, we need to expose this via REST.

I created a RESTWS-766 with a proposed design and endpoints.

I want to get ideas from others.

Regards,

Wyclif

1 Like

Looks good to me. :slight_smile:

Sure , its a good idea

@wyclif,

Considering your proposed design:

Rebuild entire index POST /openmrs/ws/rest/v1/searchindex

Rebuild index for a given type (type will be matched by resource name)

POST /openmrs/ws/rest/v1/searchindex/resourceName

Rebuild index for a specific object (uuid of the resource)

POST /openmrs/ws/rest/v1/searchindex/resourceName/uuid

In a REST API, resources are nouns (things). Your proposed design looks kind of like a verb (method). I see a couple other issues as well:

  • What is a ‘SearchIndex` (what do I get back from a GET? Likewise, what do you get if you do an HTTP GET to any of these endpoints?
  • UUIDs are supposed to be universally unique; however, in your design you are using the same UUID for the index entries as the resources themselves.
  • How would batch indexing be handled in this design? It looks like it would require n separate HTTP calls to index n resources.

In RESTful design, another option to address these concerns would be a SearchIndexRequest to which you post a list of resources to index.

1 Like

Hi Burke,

Thanks for the feedback, yes you are right, it feels like the resource is a verb but if you think about it, this is a special resource that’s intended to trigger an action and there is already existing resources designed exactly like this e.g change password resource.

Just like some already existing resources like change password if you make a GET request when it’s not support you get an exception, something along the lines of OperationNotSupportedException, this resource only supports POST and not GET. Just like you can also ask what is a ‘change password’? It also returns nothing except status OK, though you make a good point about what the response should be, we could return true for a successful rebuild otherwise false.

That uuid is for the object to be reindexed so it will always be unique e.g. if you want to rebuild the index for a patient with uuid abc the call would be POST /openmrs/ws/rest/v1/searchindex/patient/abc

I provided 3 urls, the first /openmrs/ws/rest/v1/searchindex is the one that triggers rebuilding the entire index so there would be no need to make multiple calls. If one wants to rebuild the index for say 100 concepts, one would have to either rebuild the entire index for all concepts or make 100 calls, one for each concept, this is not any different from say updating 100 concepts via rest where still you would have to make 100 update requests, one for each single concept. We can support batch rebuild for many concepts by supporting a comma separated list of concepts as the body e.g. /openmrs/ws/rest/v1/searchindex/concept with body as uuid1,uuid2,uuid3

Regards,

Wyclif

I often refer back to Prakash’s great post on REST API Design, which describes reification (making abstract things concrete).

That’s called a method. It’s easy for Java developers like us to think this way, but it’s not RESTful design. Can you find a similar example in GitHub’s API?

Consistency is a good thing; however, consistently “special” isn’t as good of a thing. :slightly_smiling_face:

These are more indicators that our design could be improved. Having resource that you can’t get is unexpected behavior in a REST API.

Yes. I understand. In RESTful design, paths are supposed to refer to resources and resources are not supposed to have more than one path to them. The pattern of /some/path/:UUID is supposed to be the path to the resource with that UUID, so having two different paths ending in the same UUID is an indicator of a design problem.

Could we treat /searchindex as a search index update request (or even rename it to /searchindexupdate)? You could still implement essentially the same thing, but it would be:

POST /searchindexupdate?target=/patient/:UUID

which could also be supported as:

POST /searchindexupdate
[
  "/patient/:UUID"
]

which would avoid the issues above, naturally allow for batch requests, and leave open the options in the future to interact with update requests (e.g., post a request to update 500 objects which returns /searchindexrequest/:UUID, which is a resource that can be monitored for status updates).

1 Like

POST /searchindexupdate

[
  "/patient/:UUID"
]

That sounds fine to me, actually I would prefer this to update a patient’s index

POST /searchindexupdate

{
  resource: "patient",
  uuid: "abc"
}

And this to update the index for patients

POST /searchindexupdate

{
  resource: "patient"
}

There is problem as of 2.0.x, root entities like Concept and Person are not annotated with @Indexed, this makes hibernate search to fail if one attempts to rebuild the index for Concept or Patient classes, it looks like one needs to rebuild the index for ConceptName class in case of concepts and PersonName, PersonAttribute or PatientIdentifier in case of patients.

I personally don’t know if we want to expect rest clients to have to know low level implementation details like these that are very tied to the data model just to be able to rebuild the index for a given type or a single object.

I’m starting to think we might just only support rebuilding the whole index.

1 Like

I would say that let us just support rebuilding the whole index. We shall deal with rebuilding the index for a given type or single object when we come across the actual use case or need for it.

1 Like

@dkayiwa this need arises from DB Sync. When syncing, we sometimes wire over just a person name, or just a concept name, or a just handful of indexed entities like that. Rebuilding the entire index is a performance killer, we just want to be able to trigger the rebuild just for the subset of entities that are being synced. Does that make sense?

1 Like

@mksd yes it makes sense.