News and Discovery Layers

It’s a pity that our discovery layer (Primo VE) isn’t really built to handle the firehouse of news articles that are published every day. When discovery layers first started appearing a dozen or so years ago, the dream was for one search box to rule them all. The majority of the siloed content in the dozens or hundreds of databases that academic libraries subscribed to would now be discoverable in one search system. We all understood that there would likely be content left out of this dream: data in datasets, non-article business information, etc. But to me, at least, I never dreamed that newspaper articles would become a problematic thing to find in a discovery layer.

I’m assuming that the folks engineering the systems that ingest records and standardize them for discovery layers (and the product managers for these systems), realized two things very quickly. First, there would be some academic customers who would complain about the flood of non-scholarly content showing up in their search results. Second, handling all that incoming news content would slow down the systems. With Primo, the solution at some point (maybe 6-7 years ago?) was to segregate most of the news content in separately scoped search interface for news articles. For libraries that opted to set up the Newspapers Search feature in Primo, their users would see in search results a facet on the search results page that when clicked would re-run the search query in a system that only yielded news articles. Search results in the regular search scope would be mostly free of the clutter of news articles.

You can see this in the Primo VE setup at City College, which like my employer, Baruch College, is part of the City University of New York. If you run a search for “biden” in the main search scope (this link is tweaked so the newest results are first), you’ll see few news articles and you’ll also see a link in the left column of facets for “Newspapers Search.” If you click that link, you’ll see this page of search results from the Newspapers search scope (I’ve also tweaked the URL to sort by date).

If you look closely at the dates of those articles, you’ll see that the newest articles are at least two days old. You won’t find any articles that were published today or yesterday. This reflects an earlier decision made by Ex Libris to have the system take at least 72 hours to index new content. I’m sure there are very good reasons why Ex Libris had to make this decision to slow things down in terms of indexing and availability. Doing so, though, destined the discovery layer to no longer be a good alternative to the various databases the library subscribed to, as any person who needed to track down an article published in the last two days or who needed any news articles on an event from the last 72 hours would be out of luck.

If a student asks me how to find news on some new or recent topic, I tend to refer them to our news databases (Factiva, Nexis Uni, Westlaw, etc.). I’d prefer to send them to our discovery layer where they could also get books, journal articles, magazine articles, reference entries, etc., in the same interface, but if they’re asking for news, I have to send them where they are most likely to find it.