Geek asks and answers

Wicket: how to transfer a resource from a database

I am trying to create a Sitemap dynamically for a large website with thousands of pages.

Yes, I looked at creating a sitemap file offline and just statically set it, and I could do just that. But I think this is a generally useful question:

How can I transfer big data from a database to Wicket?

I followed the instructions on the Wicket SEO page and was able to get a dynamic sitemap implementation using DataProvider. But it does not scale - it runs out of memory when it calls my DataProvider method iterator()with an argument countequal to the total number of returned objects, rather than iterating over them in pieces.

I think the solution lies somewhere with WebResource/ResourceStreamingRequestTarget. But these classes expect IResourceStreamthat ultimately it comes down to an implementation InputStreamthat deals with bytes, not DB records. I would not know how to implement the method length()in such a case, as this would require visiting each record ahead of time in order to calculate the total length.

+3
source share
3 answers

I ended up using a subclass AbsractResourceStreamWriter:

public class SitemapStreamWriter extends AbstractResourceStreamWriter
{
    @Override
    public void write(OutputStream output)
    {
        String HEAD = "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n" +
                      "<urlset xmlns=\"http://www.sitemaps.org/schemas/sitemap/0.9\"\n" +
                       "        xmlns:wicket=\"http://wicket.apache.org/dtds.data/wicket-xhtml1.4-strict.dtd\">\n";
        try
        {
            output.write(HEAD.getBytes());

            // write out a <loc> entry for each of my pages here

            output.write("</urlset>\n".getBytes());
        }
        catch (IOException e)
        {
            throw new RuntimeException(e.getMessage(), e);
        }
    }
}
0
source

From the IResourceStream.length () method document:

    /**
 * Gets the size of this resource in bytes
 * 
 * TODO 1.5: rename to lengthInBytes() or let it return some sort of size object
 * 
 * @return The size of this resource in the number of bytes, or -1 if unknown
 */
long length();

So, I think it would be nice if your implementation of IResourceStream indicated that the length is unknown, and you directly transmit data when you receive records from the database.

+1
source

-1, , , .

You can also use this file as a cache, so you do not need to regenerate it every time this resource is requested (remember that you have to process simultaneous requests). You can also consider dedicated caching solutions (e.g. memcache, ehcache, etc.).

It may be cleaner than posting a static file, although static files are probably better if performance is critical.

0
source

More articles:


All Articles