Enable/Disable Soundex when search for patients

Hi,

We’ve been struggling in one of our implementations (v1.11.x) with patient search, which takes over 30 seconds now to search for a patient with the number of records reaching 350k. Searching for a way to optimize this time (note that Lucene queries aren’t being used for patient search for versions 1.x), I was exploring possibilities to disable using Soundex function. I couldn’t find any. If that indeed is the case, I think it would be good to add a variable in advanced settings to switch using Soundex on and off.

Thoughts?

1 Like

AFAIK the only place that soundex (or similar) is done (pre-Lucene) is in the namephonetics module, which is optional.

That’s what I thought as well… I think if you don’t have Name Phonetics running, then there is no soundex.

Yes, Lucene queries were introduced for patient search later. Also, I benchmarked the query with and without Soundex, and Soundex found out to be a few ms faster regardless of data growth. Here are my findings:

With Soundex Without Soundex
Records Matches Time(ms) Matches Time(ms)
100000 832 267 1171 280
200000 1032 161 1691 184
300000 1232 243 2211 263
400000 1432 315 2731 359
500000 1632 908 3251 990

The above results are from my laptop - (Core i7, 7500U, Windows 10-64 bit, MySQL v5.6)

Which did you run first? If you run two similar queries in most database engines, the second one is nearly always quicker, because of disk/table caching.

Also the time per match is much less in the ‘Without Soundex’ table.

Yes, time per match has reduced, but from end-user perspective, it has actually increased. But I get your point, I guess I should repeat the experiment with query caching disabled.

@tansentim here are the results with query caching disabled

With Soundex Without Soundex
Records Matches Time(ms) Match/ms Matches Time(ms) Match/ms
100000 298 100 2.98 635 122 5.20
150000 398 116 3.43 895 132 6.78
200000 498 163 3.06 1155 185 6.24
250000 598 179 3.34 1415 217 6.52
300000 698 232 3.01 1675 270 6.20
350000 778 870 0.89 1883 902 2.09

image

image

image

Looking at the last graph, the records retrieved per milliseconds is definitely higher without Soundex.