Geek asks and answers

Call Solr asynchronously from the Play Framework

I created the Play 2.1 Scala app. I'm not sure the best way to call Solr from the Play app:

  • To play 2 there is no Solr module.
  • AFAIK all Solr-APIs, such as SolrJ, are blocked.
  • I could turn on SolrJ's call Future, but it will also block the thread, correct?

  • Should I use the library play.api.libs.ws.WSto call Solr and use the Plays JSON support to retrieve the result (as in the example below) or is there an easier / faster way?

    val solrQuery: Future[play.api.libs.ws.Response] = WS.url("http://localhost:8983/solr/collection1/select?q=id%3A123&wt=json").get()
+3
source share
3 answers

Here's how I use WS in my side project:

val itselfNodeFuture = Statix.doParams( Statix.SolrSelectWSReq, 
    List(
    "wt"     -> "json", 
    "q"      -> "*:*",
    "fq"     -> "node_type:collection",
    "fq"     -> "id:%d".format( nodeId),
    "indent" -> "true",
    "rows"   -> "1",
    "fl"     -> "id,parent_id,title",
    "fl"     -> "date_created,date_about,date_modified")
).get()

//Use the first Await after the last future
val itselfJson = Await.result(
    itselfNodeFuture, Duration("2 sec")).json

val mainRow = (itselfJson \ "response" \ "docs").as[ Seq[JsValue]]
val mainNodeParent = (mainRow(0) \ "parent_id").as[Long]
val mainNodeTitle = (mainRow(0) \ "title").as[String]

And here the useful class that I use is especially useful doParams.

object Statix { //Noder must extend this
    def SolrSelectWSReq = WS.url("http://127.0.0.1:8080/solr-store/collection1/select/")
    def SolrUpdateWSReq = WS.url("http://127.0.0.1:8080/solr-store/collection1/update/json/")

    def doParams(request: WS.WSRequestHolder, params: List[(String, String)]) = {
        params.foldLeft( request){
            (wsReq, tuple) => wsReq.withQueryString( tuple)}}
}
+2

. , , , .

, , , .

+1

. , . , SolrJ. SolrQuery QueryResponse .

, . SolrQuery . "wt" "". , SolrJ .

val sq = new SolrQuery()
sq.set("wt", "javabin")
...

SolrQuery -, WS. ( , (, IDE). , , .)

import scala.collection.JavaConverters._

def solrQueryToForm(sq: SolrQuery): Map[String, Seq[String]] = {
  sq.getParameterNames.asScala.foldLeft(Map.empty[String, Seq[String]]) {
    case (m, n) =>
      m + (n -> sq.getParams(n))
  }
}

(.. "/select" ), , SolrQuery

def solrEndpoint(sq: SolrQuery): String = {
  val coll = sq.get("collection", defaultCollection)
  val hand = Option(sq.getRequestHandler).getOrElse(defaultHandler)
  formSolrEndpoint(solrUrl, coll, hand)
}

def formSolrEndpoint(base: String, collection: String, handler: String): String = {
  val sb = new StringBuilder(base)
  if (sb.last != '/') sb.append('/')
  sb.append(collection)
  if (!handler.startsWith("/")) sb.append('/')
  sb.append(handler)
  sb.result()
}

WSResponse QueryResponse

import com.ning.http.client.{Response => ACHResponse}

def wsResponseToQueryResponse(wsResponse: WSResponse)(implicit ctx: ExecutionContext): QueryResponse = {
  val jbcUnmarshal = {
    val rbis = wsResponse.underlying[ACHResponse].getResponseBodyAsStream

    try {
      new JavaBinCodec().unmarshal(rbis)
    }
    finally {
      if (rbis != null)
        rbis.close()
    }
  }

  // p1: SolrJ pulls the same cast
  // p2: We didn't use a SolrServer to chat with Solr so cannot provide it to QueryResponse
  new QueryResponse(jbcUnmarshal.asInstanceOf[NamedList[Object]], null)
}

Solr, async WS.

def query(sq: SolrQuery)(implicit ctx: ExecutionContext): Future[QueryResponse] = {
  val sqstar = sq.getCopy
  sqstar.set("wt", "javabin")

  WS.url(solrEndpoint(sqstar))
    .post(solrQueryToForm(sqstar))
    .map(wsResponseToQueryResponse)
}

Since Play now publishes the webservice code as a separate jar, this means that almost any project should be able to request Solr asynchronously. Hope this is helpful.

+1
source

More articles:


All Articles