From nutch-commits-returncassandrawright@example.org Mon Feb 01 10:00:15 2010 Return-Path: <nutch-commits-returncassandrawright@example.org> Delivered-To: cassandrawright@example.org Received: (qmail 61131 invoked from network); 1 Feb 2010 10:00:15 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 1 Feb 2010 10:00:15 -0000 Received: (qmail 42462 invoked by uid 500); 1 Feb 2010 10:00:15 -0000 Delivered-To: cassandrawright@example.org Received: (qmail 42411 invoked by uid 500); 1 Feb 2010 10:00:14 -0000 Mailing-List: contact cassandrawright@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:cassandrawright@example.org> List-Unsubscribe: <mailto:cassandrawright@example.org> List-Post: <mailto:cassandrawright@example.org> List-Id: <nutch-commits.lucene.apache.org> Reply-To: cassandrawright@example.org Delivered-To: mailing list cassandrawright@example.org Received: (qmail 42402 invoked by uid 99); 1 Feb 2010 10:00:14 -0000 Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 01 Feb 2010 10:00:14 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO eris.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 01 Feb 2010 10:00:12 +0000 Received: by eris.apache.org (Postfix, from userid 65534) id 4640323889FA; Mon, 1 Feb 2010 09:59:51 +0000 (UTC) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: svn commit: r905228 - in /lucene/nutch/trunk/lib: tika-core-0.5.jar tika-core-0.6.jar Date: Mon, 01 Feb 2010 09:59:51 -0000 To: cassandrawright@example.org From: cassandrawright@example.org X-Mailer: svnmailer-1.0.8 Message-Id: <cassandrawright@example.org> X-Virus-Checked: Checked by ClamAV on apache.org Author: jnioche Date: Mon Feb 1 09:59:50 2010 New Revision: 905228 URL: http://svn.apache.org/viewvc?rev=905228&view=rev Log: NUTCH-781: upgrade tika to version 0.6 Added: lucene/nutch/trunk/lib/tika-core-0.6.jar (with props) Removed: lucene/nutch/trunk/lib/tika-core-0.5.jar Added: lucene/nutch/trunk/lib/tika-core-0.6.jar URL: http://svn.apache.org/viewvc/lucene/nutch/trunk/lib/tika-core-0.6.jar?rev=905228&view=auto ============================================================================== Binary file - no diff available. Propchange: lucene/nutch/trunk/lib/tika-core-0.6.jar ------------------------------------------------------------------------------ svn:mime-type = application/octet-stream From nutch-commits-returncassandrawright@example.org Mon Feb 01 10:03:31 2010 Return-Path: <nutch-commits-returncassandrawright@example.org> Delivered-To: cassandrawright@example.org Received: (qmail 62680 invoked from network); 1 Feb 2010 10:03:31 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 1 Feb 2010 10:03:31 -0000 Received: (qmail 46629 invoked by uid 500); 1 Feb 2010 10:03:30 -0000 Delivered-To: cassandrawright@example.org Received: (qmail 46575 invoked by uid 500); 1 Feb 2010 10:03:30 -0000 Mailing-List: contact cassandrawright@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:cassandrawright@example.org> List-Unsubscribe: <mailto:cassandrawright@example.org> List-Post: <mailto:cassandrawright@example.org> List-Id: <nutch-commits.lucene.apache.org> Reply-To: cassandrawright@example.org Delivered-To: mailing list cassandrawright@example.org Received: (qmail 46565 invoked by uid 99); 1 Feb 2010 10:03:30 -0000 Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 01 Feb 2010 10:03:30 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO eris.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 01 Feb 2010 10:03:28 +0000 Received: by eris.apache.org (Postfix, from userid 65534) id DDD8123888BD; Mon, 1 Feb 2010 10:03:07 +0000 (UTC) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: svn commit: r905229 - /lucene/nutch/trunk/CHANGES.txt Date: Mon, 01 Feb 2010 10:03:07 -0000 To: cassandrawright@example.org From: cassandrawright@example.org X-Mailer: svnmailer-1.0.8 Message-Id: <cassandrawright@example.org> X-Virus-Checked: Checked by ClamAV on apache.org Author: jnioche Date: Mon Feb 1 10:03:07 2010 New Revision: 905229 URL: http://svn.apache.org/viewvc?rev=905229&view=rev Log: NUTCH-781: upgrade tika to version 0.6 Modified: lucene/nutch/trunk/CHANGES.txt Modified: lucene/nutch/trunk/CHANGES.txt URL: http://svn.apache.org/viewvc/lucene/nutch/trunk/CHANGES.txt?rev=905229&r1=905228&r2=905229&view=diff ============================================================================== --- lucene/nutch/trunk/CHANGES.txt (original) +++ lucene/nutch/trunk/CHANGES.txt Mon Feb 1 10:03:07 2010 @@ -2,6 +2,8 @@ Unreleased Changes +* NUTCH-781 Update Tika to v0.6 (jnioche) + * NUTCH-269 CrawlDbReducer: OOME because no upper-bound on inlinks count (stack + jnioche) * NUTCH-655 Injecting Crawl metadata (jnioche) From nutch-commits-returncassandrawright@example.org Mon Feb 01 20:48:02 2010 Return-Path: <nutch-commits-returncassandrawright@example.org> Delivered-To: cassandrawright@example.org Received: (qmail 98142 invoked from network); 1 Feb 2010 20:48:02 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 1 Feb 2010 20:48:02 -0000 Received: (qmail 12546 invoked by uid 500); 1 Feb 2010 20:48:02 -0000 Delivered-To: cassandrawright@example.org Received: (qmail 12491 invoked by uid 500); 1 Feb 2010 20:48:02 -0000 Mailing-List: contact cassandrawright@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:cassandrawright@example.org> List-Unsubscribe: <mailto:cassandrawright@example.org> List-Post: <mailto:cassandrawright@example.org> List-Id: <nutch-commits.lucene.apache.org> Reply-To: cassandrawright@example.org Delivered-To: mailing list cassandrawright@example.org Received: (qmail 12482 invoked by uid 99); 1 Feb 2010 20:48:02 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 01 Feb 2010 20:48:02 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO eris.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 01 Feb 2010 20:47:55 +0000 Received: by eris.apache.org (Postfix, from userid 65534) id 3958223888D1; Mon, 1 Feb 2010 20:47:35 +0000 (UTC) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: svn commit: r905410 - in /lucene/nutch/trunk: ./ src/java/org/apache/nutch/searcher/ src/test/org/apache/nutch/searcher/ src/web/jsp/ Date: Mon, 01 Feb 2010 20:47:35 -0000 To: cassandrawright@example.org From: cassandrawright@example.org X-Mailer: svnmailer-1.0.8 Message-Id: <cassandrawright@example.org> Author: siren Date: Mon Feb 1 20:47:34 2010 New Revision: 905410 URL: http://svn.apache.org/viewvc?rev=905410&view=rev Log: NUTCH-775 Enhance Searcher interface Added: lucene/nutch/trunk/src/java/org/apache/nutch/searcher/QueryParams.java lucene/nutch/trunk/src/test/org/apache/nutch/searcher/QueryParamsTest.java Modified: lucene/nutch/trunk/CHANGES.txt lucene/nutch/trunk/src/java/org/apache/nutch/searcher/DistributedSearchBean.java lucene/nutch/trunk/src/java/org/apache/nutch/searcher/IndexSearcher.java lucene/nutch/trunk/src/java/org/apache/nutch/searcher/LuceneSearchBean.java lucene/nutch/trunk/src/java/org/apache/nutch/searcher/NutchBean.java lucene/nutch/trunk/src/java/org/apache/nutch/searcher/Query.java lucene/nutch/trunk/src/java/org/apache/nutch/searcher/Searcher.java lucene/nutch/trunk/src/java/org/apache/nutch/searcher/SolrSearchBean.java lucene/nutch/trunk/src/web/jsp/search.jsp Modified: lucene/nutch/trunk/CHANGES.txt URL: http://svn.apache.org/viewvc/lucene/nutch/trunk/CHANGES.txt?rev=905410&r1=905409&r2=905410&view=diff ============================================================================== --- lucene/nutch/trunk/CHANGES.txt (original) +++ lucene/nutch/trunk/CHANGES.txt Mon Feb 1 20:47:34 2010 @@ -2,6 +2,8 @@ Unreleased Changes +* NUTCH-775 Enhance searcher interface (siren) + * NUTCH-781 Update Tika to v0.6 (jnioche) * NUTCH-269 CrawlDbReducer: OOME because no upper-bound on inlinks count (stack + jnioche) Modified: lucene/nutch/trunk/src/java/org/apache/nutch/searcher/DistributedSearchBean.java URL: http://svn.apache.org/viewvc/lucene/nutch/trunk/src/java/org/apache/nutch/searcher/DistributedSearchBean.java?rev=905410&r1=905409&r2=905410&view=diff ============================================================================== --- lucene/nutch/trunk/src/java/org/apache/nutch/searcher/DistributedSearchBean.java (original) +++ lucene/nutch/trunk/src/java/org/apache/nutch/searcher/DistributedSearchBean.java Mon Feb 1 20:47:34 2010 @@ -49,10 +49,6 @@ private int id; private Query query; - private int numHits; - private String dedupField; - private String sortField; - private boolean reverse; public SearchTask(int id) { this.id = id; @@ -62,16 +58,20 @@ if (!liveServers[id]) { return null; } - return beans[id].search(query, numHits, dedupField, sortField, reverse); + return beans[id].search(query); } + /** + * @deprecated since 1.1, use {@link #setSearchArgs(Query)} instead + */ public void setSearchArgs(Query query, int numHits, String dedupField, String sortField, boolean reverse) { this.query = query; - this.numHits = numHits; - this.dedupField = dedupField; - this.sortField = sortField; - this.reverse = reverse; + query.setParams(new QueryParams(numHits, QueryParams.DEFAULT_MAX_HITS_PER_DUP, dedupField, sortField, reverse)); + } + + private void setSearchArgs(Query query) { + this.query = query; } } @@ -199,12 +199,10 @@ return beans[hit.getIndexNo()].getExplanation(query, hit); } - public Hits search(Query query, int numHits, String dedupField, - String sortField, boolean reverse) throws IOException { - + @Override + public Hits search(Query query) throws IOException { for (Callable<Hits> task : searchTasks) { - ((SearchTask)task).setSearchArgs(query, numHits, dedupField, sortField, - reverse); + ((SearchTask)task).setSearchArgs(query); } List<Future<Hits>> allHits; @@ -216,10 +214,12 @@ } PriorityQueue<Hit> queue; // cull top hits from results - if (sortField == null || reverse) { - queue = new PriorityQueue<Hit>(numHits); + if (query.getParams().getSortField() == null + || query.getParams().isReverse()) { + queue = new PriorityQueue<Hit>(query.getParams().getNumHits()); } else { - queue = new PriorityQueue<Hit>(numHits, new Comparator<Hit>() { + queue = new PriorityQueue<Hit>(query.getParams().getNumHits(), + new Comparator<Hit>() { public int compare(Hit h1, Hit h2) { return h2.compareTo(h1); // reverse natural order } @@ -251,7 +251,8 @@ Hit newHit = new Hit(i, hit.getUniqueKey(), hit.getSortValue(), hit.getDedupValue()); queue.add(newHit); - if (queue.size() > numHits) { // if hit queue overfull + if (queue.size() > query.getParams().getNumHits()) { + // if hit queue overfull queue.remove(); } } @@ -265,6 +266,15 @@ return new Hits(totalHits, culledResults); } + @Override + @Deprecated + public Hits search(Query query, int numHits, String dedupField, + String sortField, boolean reverse) throws IOException { + + query.setParams(new QueryParams(numHits, QueryParams.DEFAULT_MAX_HITS_PER_DUP, dedupField, sortField, reverse)); + return search(query); + } + public void close() throws IOException { executor.shutdown(); pingService.shutdown(); Modified: lucene/nutch/trunk/src/java/org/apache/nutch/searcher/IndexSearcher.java URL: http://svn.apache.org/viewvc/lucene/nutch/trunk/src/java/org/apache/nutch/searcher/IndexSearcher.java?rev=905410&r1=905409&r2=905410&view=diff ============================================================================== --- lucene/nutch/trunk/src/java/org/apache/nutch/searcher/IndexSearcher.java (original) +++ lucene/nutch/trunk/src/java/org/apache/nutch/searcher/IndexSearcher.java Mon Feb 1 20:47:34 2010 @@ -89,18 +89,28 @@ } } + @Override + @Deprecated public Hits search(Query query, int numHits, String dedupField, String sortField, boolean reverse) throws IOException { + query.setParams(new QueryParams(numHits, + QueryParams.DEFAULT_MAX_HITS_PER_DUP, dedupField, sortField, reverse)); + return search(query); + } + + @Override + public Hits search(Query query) throws IOException { org.apache.lucene.search.BooleanQuery luceneQuery = this.queryFilters.filter(query); - return translateHits - (optimizer.optimize(luceneQuery, luceneSearcher, numHits, - sortField, reverse), - dedupField, sortField); + return translateHits(optimizer.optimize(luceneQuery, luceneSearcher, query + .getParams().getNumHits(), query.getParams().getSortField(), query + .getParams().isReverse()), query.getParams().getDedupField(), query + .getParams().getSortField()); } + public String getExplanation(Query query, Hit hit) throws IOException { return luceneSearcher.explain(this.queryFilters.filter(query), Integer.valueOf(hit.getUniqueKey())).toHtml(); Modified: lucene/nutch/trunk/src/java/org/apache/nutch/searcher/LuceneSearchBean.java URL: http://svn.apache.org/viewvc/lucene/nutch/trunk/src/java/org/apache/nutch/searcher/LuceneSearchBean.java?rev=905410&r1=905409&r2=905410&view=diff ============================================================================== --- lucene/nutch/trunk/src/java/org/apache/nutch/searcher/LuceneSearchBean.java (original) +++ lucene/nutch/trunk/src/java/org/apache/nutch/searcher/LuceneSearchBean.java Mon Feb 1 20:47:34 2010 @@ -78,10 +78,19 @@ } } + + @Override + @Deprecated public Hits search(Query query, int numHits, String dedupField, String sortField, boolean reverse) throws IOException { - return searcher.search(query, numHits, dedupField, sortField, reverse); + query.setParams(new QueryParams(numHits, QueryParams.DEFAULT_MAX_HITS_PER_DUP, dedupField, sortField, reverse)); + return searcher.search(query); + } + + @Override + public Hits search(Query query) throws IOException { + return searcher.search(query); } public String getExplanation(Query query, Hit hit) throws IOException { Modified: lucene/nutch/trunk/src/java/org/apache/nutch/searcher/NutchBean.java URL: http://svn.apache.org/viewvc/lucene/nutch/trunk/src/java/org/apache/nutch/searcher/NutchBean.java?rev=905410&r1=905409&r2=905410&view=diff ============================================================================== --- lucene/nutch/trunk/src/java/org/apache/nutch/searcher/NutchBean.java (original) +++ lucene/nutch/trunk/src/java/org/apache/nutch/searcher/NutchBean.java Mon Feb 1 20:47:34 2010 @@ -151,86 +151,35 @@ return segmentBean.getSegmentNames(); } + /** + * @deprecated since 1.1, use {@link #search(Query)} instead + */ public Hits search(Query query, int numHits) throws IOException { return search(query, numHits, null, null, false); } + /** + * @deprecated since 1.1, use {@link #search(Query)} instead + */ public Hits search(Query query, int numHits, String dedupField, String sortField, boolean reverse) throws IOException { - return searchBean.search(query, numHits, dedupField, sortField, reverse); - } - - @SuppressWarnings("serial") - private class DupHits extends ArrayList<Hit> { - private boolean maxSizeExceeded; - } - - /** Search for pages matching a query, eliminating excessive hits from the - * same site. Hits after the first <code>maxHitsPerDup</code> from the same - * site are removed from results. The remaining hits have {@link - * Hit#moreFromDupExcluded()} set. <p> If maxHitsPerDup is zero then all - * hits are returned. - * - * @param query query - * @param numHits number of requested hits - * @param maxHitsPerDup the maximum hits returned with matching values, or zero - * @return Hits the matching hits - * @throws IOException - */ - public Hits search(Query query, int numHits, int maxHitsPerDup) - throws IOException { - return search(query, numHits, maxHitsPerDup, "site", null, false); - } - - /** Search for pages matching a query, eliminating excessive hits with - * matching values for a named field. Hits after the first - * <code>maxHitsPerDup</code> are removed from results. The remaining hits - * have {@link Hit#moreFromDupExcluded()} set. <p> If maxHitsPerDup is zero - * then all hits are returned. - * - * @param query query - * @param numHits number of requested hits - * @param maxHitsPerDup the maximum hits returned with matching values, or zero - * @param dedupField field name to check for duplicates - * @return Hits the matching hits - * @throws IOException - */ - public Hits search(Query query, int numHits, - int maxHitsPerDup, String dedupField) - throws IOException { - return search(query, numHits, maxHitsPerDup, dedupField, null, false); + query.getParams().initFrom(numHits, QueryParams.DEFAULT_MAX_HITS_PER_DUP, dedupField, sortField, reverse); + return search(query); } - /** Search for pages matching a query, eliminating excessive hits with - * matching values for a named field. Hits after the first - * <code>maxHitsPerDup</code> are removed from results. The remaining hits - * have {@link Hit#moreFromDupExcluded()} set. <p> If maxHitsPerDup is zero - * then all hits are returned. - * - * @param query query - * @param numHits number of requested hits - * @param maxHitsPerDup the maximum hits returned with matching values, or zero - * @param dedupField field name to check for duplicates - * @param sortField Field to sort on (or null if no sorting). - * @param reverse True if we are to reverse sort by <code>sortField</code>. - * @return Hits the matching hits - * @throws IOException - */ - public Hits search(Query query, int numHits, - int maxHitsPerDup, String dedupField, - String sortField, boolean reverse) - throws IOException { - if (maxHitsPerDup <= 0) // disable dup checking - return search(query, numHits, dedupField, sortField, reverse); + + @Override + public Hits search(Query query) throws IOException { + if (query.getParams().getMaxHitsPerDup() <= 0) // disable dup checking + return searchBean.search(query); final float rawHitsFactor = this.conf.getFloat("searcher.hostgrouping.rawhits.factor", 2.0f); - int numHitsRaw = (int)(numHits * rawHitsFactor); + int numHitsRaw = (int)(query.getParams().getNumHits() * rawHitsFactor); if (LOG.isInfoEnabled()) { LOG.info("searching for "+numHitsRaw+" raw hits"); } - Hits hits = searchBean.search(query, numHitsRaw, - dedupField, sortField, reverse); + Hits hits = searchBean.search(query); final long total = hits.getTotal(); final Map<String, DupHits> dupToHits = new HashMap<String, DupHits>(); final List<Hit> resultList = new ArrayList<Hit>(); @@ -246,14 +195,13 @@ if (i == MAX_PROHIBITED_TERMS) break; optQuery.addProhibitedTerm(excludedValues.get(i), - dedupField); + query.getParams().getDedupField()); } numHitsRaw = (int)(numHitsRaw * rawHitsFactor); if (LOG.isInfoEnabled()) { LOG.info("re-searching for "+numHitsRaw+" raw hits, query: "+optQuery); } - hits = searchBean.search(optQuery, numHitsRaw, - dedupField, sortField, reverse); + hits = searchBean.search(optQuery); if (LOG.isInfoEnabled()) { LOG.info("found "+hits.getTotal()+" raw hits"); } @@ -273,7 +221,7 @@ dupToHits.put(value, dupHits = new DupHits()); // does this hit exceed maxHitsPerDup? - if (dupHits.size() == maxHitsPerDup) { // yes -- ignore the hit + if (dupHits.size() == query.getParams().getMaxHitsPerDup()) { // yes -- ignore the hit if (!dupHits.maxSizeExceeded) { // mark prior hits with moreFromDupExcluded @@ -292,7 +240,7 @@ // are we done? // we need to find one more than asked for, so that we can tell if // there are more hits to be shown - if (resultList.size() > numHits) + if (resultList.size() > query.getParams().getNumHits()) break; } } @@ -304,6 +252,73 @@ return results; } + @SuppressWarnings("serial") + private class DupHits extends ArrayList<Hit> { + private boolean maxSizeExceeded; + } + + /** Search for pages matching a query, eliminating excessive hits from the + * same site. Hits after the first <code>maxHitsPerDup</code> from the same + * site are removed from results. The remaining hits have {@link + * Hit#moreFromDupExcluded()} set. <p> If maxHitsPerDup is zero then all + * hits are returned. + * + * @param query query + * @param numHits number of requested hits + * @param maxHitsPerDup the maximum hits returned with matching values, or zero + * @return Hits the matching hits + * @throws IOException + * @deprecated since 1.1, use {@link #search(Query)} instead + * + */ + public Hits search(Query query, int numHits, int maxHitsPerDup) + throws IOException { + return search(query, numHits, maxHitsPerDup, "site", null, false); + } + + /** Search for pages matching a query, eliminating excessive hits with + * matching values for a named field. Hits after the first + * <code>maxHitsPerDup</code> are removed from results. The remaining hits + * have {@link Hit#moreFromDupExcluded()} set. <p> If maxHitsPerDup is zero + * then all hits are returned. + * + * @param query query + * @param numHits number of requested hits + * @param maxHitsPerDup the maximum hits returned with matching values, or zero + * @param dedupField field name to check for duplicates + * @return Hits the matching hits + * @throws IOException + * @deprecated since 1.1, use {@link #search(Query)} instead + */ + public Hits search(Query query, int numHits, + int maxHitsPerDup, String dedupField) + throws IOException { + return search(query, numHits, maxHitsPerDup, dedupField, null, false); + } + /** Search for pages matching a query, eliminating excessive hits with + * matching values for a named field. Hits after the first + * <code>maxHitsPerDup</code> are removed from results. The remaining hits + * have {@link Hit#moreFromDupExcluded()} set. <p> If maxHitsPerDup is zero + * then all hits are returned. + * + * @param query query + * @param numHits number of requested hits + * @param maxHitsPerDup the maximum hits returned with matching values, or zero + * @param dedupField field name to check for duplicates + * @param sortField Field to sort on (or null if no sorting). + * @param reverse True if we are to reverse sort by <code>sortField</code>. + * @return Hits the matching hits + * @throws IOException + * @deprecated since 1.1, use {@link #search(Query)} instead + */ + public Hits search(Query query, int numHits, + int maxHitsPerDup, String dedupField, + String sortField, boolean reverse) + throws IOException { + query.setParams(new QueryParams(numHits, maxHitsPerDup, dedupField, sortField, reverse)); + return search(query); + } + public String getExplanation(Query query, Hit hit) throws IOException { return searchBean.getExplanation(query, hit); @@ -374,9 +389,10 @@ final NutchBean bean = new NutchBean(conf); try { final Query query = Query.parse(args[0], conf); - final Hits hits = bean.search(query, 10); + query.getParams().setMaxHitsPerDup(0); + final Hits hits = bean.search(query); System.out.println("Total hits: " + hits.getTotal()); - final int length = (int)Math.min(hits.getTotal(), 10); + final int length = (int)Math.min(hits.getLength(), 10); final Hit[] show = hits.getHits(0, length); final HitDetails[] details = bean.getDetails(show); final Summary[] summaries = bean.getSummary(details, query); Modified: lucene/nutch/trunk/src/java/org/apache/nutch/searcher/Query.java URL: http://svn.apache.org/viewvc/lucene/nutch/trunk/src/java/org/apache/nutch/searcher/Query.java?rev=905410&r1=905409&r2=905410&view=diff ============================================================================== --- lucene/nutch/trunk/src/java/org/apache/nutch/searcher/Query.java (original) +++ lucene/nutch/trunk/src/java/org/apache/nutch/searcher/Query.java Mon Feb 1 20:47:34 2010 @@ -41,6 +41,16 @@ public final class Query implements Writable, Cloneable, Configurable { public static final Log LOG = LogFactory.getLog(Query.class); + private QueryParams params = new QueryParams(); + + public void setParams(QueryParams context) { + this.params = context; + } + + public QueryParams getParams() { + return params; + } + /** A query clause. */ public static class Clause implements Cloneable { public static final String DEFAULT_FIELD = "DEFAULT"; @@ -362,6 +372,7 @@ out.writeByte(clauses.size()); for (int i = 0; i < clauses.size(); i++) clauses.get(i).write(out); + params.write(out); } public static Query read(DataInput in, Configuration conf) throws IOException { @@ -375,6 +386,8 @@ int length = in.readByte(); for (int i = 0; i < length; i++) clauses.add(Clause.read(in, this.conf)); + + params.readFields(in); } public String toString() { @@ -390,7 +403,7 @@ public boolean equals(Object o) { if (!(o instanceof Query)) return false; Query other = (Query)o; - return this.clauses.equals(other.clauses); + return this.clauses.equals(other.clauses) && this.params.equals(other.params); } public int hashCode() { Added: lucene/nutch/trunk/src/java/org/apache/nutch/searcher/QueryParams.java URL: http://svn.apache.org/viewvc/lucene/nutch/trunk/src/java/org/apache/nutch/searcher/QueryParams.java?rev=905410&view=auto ============================================================================== --- lucene/nutch/trunk/src/java/org/apache/nutch/searcher/QueryParams.java (added) +++ lucene/nutch/trunk/src/java/org/apache/nutch/searcher/QueryParams.java Mon Feb 1 20:47:34 2010 @@ -0,0 +1,138 @@ +package org.apache.nutch.searcher; + +import java.io.DataInput; +import java.io.DataOutput; +import java.io.IOException; + +import org.apache.hadoop.io.Writable; +import org.apache.hadoop.io.WritableUtils; +import org.apache.nutch.metadata.Metadata; + +/** + * Query context object that describes the context of the query. + */ +public class QueryParams implements Writable { + + public static final String DEFAULT_DEDUP_FIELD = "site"; + public static final int DEFAULT_MAX_HITS_PER_DUP = 2; + public static final int DEFAULT_NUM_HITS = 10; + public static final boolean DEFAULT_REVERSE = false; + + private Metadata metadata = new Metadata(); + + public void setNumHits(int numHits) { + this.numHits = numHits; + } + + public void setMaxHitsPerDup(int maxHitsPerDup) { + this.maxHitsPerDup = maxHitsPerDup; + } + + public void setDedupField(String dedupField) { + this.dedupField = dedupField; + } + + public void setSortField(String sortField) { + this.sortField = sortField; + } + + public void setReverse(boolean reverse) { + this.reverse = reverse; + } + + private int numHits; + private int maxHitsPerDup; + private String dedupField; + private String sortField; + private boolean reverse; + + public QueryParams() { + setNumHits(DEFAULT_NUM_HITS); + setMaxHitsPerDup(DEFAULT_MAX_HITS_PER_DUP); + setDedupField(DEFAULT_DEDUP_FIELD); + setSortField(sortField); + setReverse(false); + + } + + public QueryParams(int numHits, int maxHitsPerDup, String dedupField, + String sortField, boolean reverse) { + initFrom(numHits, maxHitsPerDup, dedupField, sortField, reverse); + } + + public void initFrom(int numHits, int maxHitsPerDup, String dedupField, + String sortField, boolean reverse) { + setNumHits(numHits); + setMaxHitsPerDup(maxHitsPerDup); + setDedupField(dedupField); + setSortField(sortField); + setReverse(reverse); + } + + public int getNumHits() { + return numHits; + } + + public int getMaxHitsPerDup() { + return maxHitsPerDup; + } + + public String getDedupField() { + return dedupField; + } + + public String getSortField() { + return sortField; + } + + public boolean isReverse() { + return reverse; + } + + public String get(String name) { + return metadata.get(name); + } + + public void put(String name, String value) { + metadata.set(name, value); + } + + @Override + public void readFields(DataInput input) throws IOException { + metadata.readFields(input); + numHits = WritableUtils.readVInt(input); + maxHitsPerDup = WritableUtils.readVInt(input); + dedupField = WritableUtils.readString(input); + sortField = WritableUtils.readString(input); + reverse = input.readBoolean(); + } + + @Override + public void write(DataOutput output) throws IOException { + metadata.write(output); + WritableUtils.writeVInt(output, numHits); + WritableUtils.writeVInt(output, maxHitsPerDup); + WritableUtils.writeString(output, dedupField); + WritableUtils.writeString(output, sortField); + output.writeBoolean(reverse); + } + + @Override + public boolean equals(Object obj) { + if (obj instanceof QueryParams) { + + QueryParams other = (QueryParams) obj; + return other.numHits == this.numHits + && other.metadata.equals(this.metadata) + && other.reverse == this.reverse + && other.maxHitsPerDup == this.maxHitsPerDup + && ((other.dedupField != null && other.dedupField + .equals(this.dedupField)) || (other.dedupField == null && this.dedupField == null)) + && ((other.sortField != null && other.sortField + .equals(this.sortField)) || (other.sortField == null && this.sortField == null)); + + } else { + return false; + } + } +} Modified: lucene/nutch/trunk/src/java/org/apache/nutch/searcher/Searcher.java URL: http://svn.apache.org/viewvc/lucene/nutch/trunk/src/java/org/apache/nutch/searcher/Searcher.java?rev=905410&r1=905409&r2=905410&view=diff ============================================================================== --- lucene/nutch/trunk/src/java/org/apache/nutch/searcher/Searcher.java (original) +++ lucene/nutch/trunk/src/java/org/apache/nutch/searcher/Searcher.java Mon Feb 1 20:47:34 2010 @@ -23,10 +23,20 @@ /** Service that searches. */ public interface Searcher extends Closeable { - /** Return the top-scoring hits for a query. */ - Hits search(Query query, int numHits, - String dedupField, String sortField, boolean reverse) - throws IOException; + /** + * Return the top-scoring hits for a query. + * + * @deprecated since 1.1, use {@link #search(Query)} instead. + * */ + Hits search(Query query, int numHits, String dedupField, String sortField, + boolean reverse) throws IOException; + + /** + * Return the top-scoring hits for a query. + * + * @since 1.1 + */ + Hits search(Query query) throws IOException; /** Return an HTML-formatted explanation of how a query scored. */ String getExplanation(Query query, Hit hit) throws IOException; Modified: lucene/nutch/trunk/src/java/org/apache/nutch/searcher/SolrSearchBean.java URL: http://svn.apache.org/viewvc/lucene/nutch/trunk/src/java/org/apache/nutch/searcher/SolrSearchBean.java?rev=905410&r1=905409&r2=905410&view=diff ============================================================================== --- lucene/nutch/trunk/src/java/org/apache/nutch/searcher/SolrSearchBean.java (original) +++ lucene/nutch/trunk/src/java/org/apache/nutch/searcher/SolrSearchBean.java Mon Feb 1 20:47:34 2010 @@ -68,24 +68,23 @@ return "SOLR backend does not support explanations yet."; } - @SuppressWarnings("unchecked") - public Hits search(Query query, int numHits, String dedupField, - String sortField, boolean reverse) - throws IOException { - + + public Hits search(Query query) throws IOException { // filter query string final BooleanQuery bQuery = filters.filter(query); final SolrQuery solrQuery = new SolrQuery(stringify(bQuery)); - solrQuery.setRows(numHits); + solrQuery.setRows(query.getParams().getNumHits()); - if (sortField == null) { - solrQuery.setFields(dedupField, "score", searchUID); - sortField = "score"; + if (query.getParams().getSortField() == null) { + solrQuery.setFields(query.getParams().getDedupField(), "score", searchUID); + query.getParams().setSortField("score"); } else { - solrQuery.setFields(dedupField, sortField, searchUID); - solrQuery.setSortField(sortField, reverse ? ORDER.asc : ORDER.desc); + solrQuery.setFields(query.getParams().getDedupField(), query + .getParams().getSortField(), searchUID); + solrQuery.setSortField(query.getParams().getSortField(), query + .getParams().isReverse() ? ORDER.asc : ORDER.desc); } QueryResponse response; @@ -101,7 +100,7 @@ for (int i = 0; i < hitArr.length; i++) { final SolrDocument solrDoc = docList.get(i); - final Object raw = solrDoc.getFirstValue(sortField); + final Object raw = solrDoc.getFirstValue(query.getParams().getSortField()); WritableComparable sortValue; if (raw instanceof Integer) { @@ -116,7 +115,7 @@ throw new RuntimeException("Unknown sort value type!"); } - final String dedupValue = (String) solrDoc.getFirstValue(dedupField); + final String dedupValue = (String) solrDoc.getFirstValue(query.getParams().getDedupField()); final String uniqueKey = (String )solrDoc.getFirstValue(searchUID); @@ -126,6 +125,18 @@ return new Hits(docList.getNumFound(), hitArr); } + @SuppressWarnings("unchecked") + @Deprecated + public Hits search(Query query, int numHits, String dedupField, + String sortField, boolean reverse) + throws IOException { + query.getParams().setNumHits(numHits); + query.getParams().setDedupField(dedupField); + query.getParams().setSortField(sortField); + query.getParams().setReverse(reverse); + return search(query); + } + public HitDetails getDetails(Hit hit) throws IOException { QueryResponse response; try { Added: lucene/nutch/trunk/src/test/org/apache/nutch/searcher/QueryParamsTest.java URL: http://svn.apache.org/viewvc/lucene/nutch/trunk/src/test/org/apache/nutch/searcher/QueryParamsTest.java?rev=905410&view=auto ============================================================================== --- lucene/nutch/trunk/src/test/org/apache/nutch/searcher/QueryParamsTest.java (added) +++ lucene/nutch/trunk/src/test/org/apache/nutch/searcher/QueryParamsTest.java Mon Feb 1 20:47:34 2010 @@ -0,0 +1,15 @@ +package org.apache.nutch.searcher; + +import junit.framework.TestCase; + +import org.apache.nutch.util.WritableTestUtils; + +public class QueryParamsTest extends TestCase { + + public void testWritable() throws Exception { + QueryParams context = new QueryParams(10, 2, "site", null, false); + context.put("cat", "dog"); + WritableTestUtils.testWritable(context); + } + +} Modified: lucene/nutch/trunk/src/web/jsp/search.jsp URL: http://svn.apache.org/viewvc/lucene/nutch/trunk/src/web/jsp/search.jsp?rev=905410&r1=905409&r2=905410&view=diff ============================================================================== --- lucene/nutch/trunk/src/web/jsp/search.jsp (original) +++ lucene/nutch/trunk/src/web/jsp/search.jsp Mon Feb 1 20:47:34 2010 @@ -204,8 +204,8 @@ // position.... this is good, bad?... ugly?.... Hits hits; try{ - hits = bean.search(query, start + hitsToRetrieve, hitsPerSite, "site", - sort, reverse); + query.getParams.initFrom(start + hitsToRetrieve, hitsPerSite, "site", sort, reverse); + hits = bean.search(query); } catch (IOException e){ hits = new Hits(0,new Hit[0]); } From nutch-commits-returncassandrawright@example.org Tue Feb 02 09:31:46 2010 Return-Path: <nutch-commits-returncassandrawright@example.org> Delivered-To: cassandrawright@example.org Received: (qmail 51009 invoked from network); 2 Feb 2010 09:31:46 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 2 Feb 2010 09:31:46 -0000 Received: (qmail 46720 invoked by uid 500); 2 Feb 2010 09:31:45 -0000 Delivered-To: cassandrawright@example.org Received: (qmail 46670 invoked by uid 500); 2 Feb 2010 09:31:45 -0000 Mailing-List: contact cassandrawright@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:cassandrawright@example.org> List-Unsubscribe: <mailto:cassandrawright@example.org> List-Post: <mailto:cassandrawright@example.org> List-Id: <nutch-commits.lucene.apache.org> Reply-To: cassandrawright@example.org Delivered-To: mailing list cassandrawright@example.org Received: (qmail 46646 invoked by uid 99); 2 Feb 2010 09:31:44 -0000 Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 02 Feb 2010 09:31:44 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO eris.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 02 Feb 2010 09:31:42 +0000 Received: by eris.apache.org (Postfix, from userid 65534) id 95F7123889C5; Tue, 2 Feb 2010 09:31:21 +0000 (UTC) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: svn commit: r905550 [1/2] - /lucene/nutch/trunk/conf/tika-mimetypes.xml Date: Tue, 02 Feb 2010 09:31:20 -0000 To: cassandrawright@example.org From: cassandrawright@example.org X-Mailer: svnmailer-1.0.8 Message-Id: <cassandrawright@example.org> X-Virus-Checked: Checked by ClamAV on apache.org Author: jnioche Date: Tue Feb 2 09:31:19 2010 New Revision: 905550 URL: http://svn.apache.org/viewvc?rev=905550&view=rev Log: NUTCH-781 : updated tika-mimetypes.xml Modified: lucene/nutch/trunk/conf/tika-mimetypes.xml From nutch-commits-returncassandrawright@example.org Tue Feb 02 09:32:00 2010 Return-Path: <nutch-commits-returncassandrawright@example.org> Delivered-To: cassandrawright@example.org Received: (qmail 51142 invoked from network); 2 Feb 2010 09:32:00 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 2 Feb 2010 09:32:00 -0000 Received: (qmail 46933 invoked by uid 500); 2 Feb 2010 09:32:00 -0000 Delivered-To: cassandrawright@example.org Received: (qmail 46883 invoked by uid 500); 2 Feb 2010 09:32:00 -0000 Mailing-List: contact cassandrawright@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:cassandrawright@example.org> List-Unsubscribe: <mailto:cassandrawright@example.org> List-Post: <mailto:cassandrawright@example.org> List-Id: <nutch-commits.lucene.apache.org> Reply-To: cassandrawright@example.org Delivered-To: mailing list cassandrawright@example.org Received: (qmail 46874 invoked by uid 99); 2 Feb 2010 09:32:00 -0000 Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 02 Feb 2010 09:32:00 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO eris.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 02 Feb 2010 09:31:44 +0000 Received: by eris.apache.org (Postfix, from userid 65534) id 99B7323889E2; Tue, 2 Feb 2010 09:31:21 +0000 (UTC) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: svn commit: r905550 [2/2] - /lucene/nutch/trunk/conf/tika-mimetypes.xml Date: Tue, 02 Feb 2010 09:31:20 -0000 To: cassandrawright@example.org From: cassandrawright@example.org X-Mailer: svnmailer-1.0.8 Message-Id: <cassandrawright@example.org> X-Virus-Checked: Checked by ClamAV on apache.org Modified: lucene/nutch/trunk/conf/tika-mimetypes.xml URL: http://svn.apache.org/viewvc/lucene/nutch/trunk/conf/tika-mimetypes.xml?rev=905550&r1=905549&r2=905550&view=diff ============================================================================== --- lucene/nutch/trunk/conf/tika-mimetypes.xml (original) +++ lucene/nutch/trunk/conf/tika-mimetypes.xml Tue Feb 2 09:31:19 2010 @@ -17,261 +17,1600 @@ --> <!-- Description: This xml file defines the valid mime types used by Tika. - The mime types within this file are based on the types in the mime-types.xml - file available in Apache Nutch. + The mime type data within this file is based on information from various + sources like Apache Nutch, Apache HTTP Server, the file(1) command, etc. --> <mime-info> - <mime-type type="text/plain"> - <magic priority="20"> - <match value="This is TeX," type="string" offset="0" /> - <match value="This is METAFONT," type="string" offset="0" /> - <match value="#!/" type="string" offset="0" /> - <match value="#!\ /" type="string" offset="0" /> - <match value="#!\t/" type="string" offset="0" /> - <!-- UTF-16BE BOM --> - <match value="0xfeff" type="string" offset="0"/> - <!-- UTF-16LE BOM --> - <match value="0xfffe" type="string" offset="0"/> - <!-- UTF-8 BOM --> - <match value="0xefbbbf" type="string" offset="0"/> + <mime-type type="application/activemessage"/> + <mime-type type="application/andrew-inset"> + <glob pattern="*.ez"/> + </mime-type> + <mime-type type="application/applefile"/> + <mime-type type="application/applixware"> + <glob pattern="*.aw"/> + </mime-type> + + <mime-type type="application/atom+xml"> + <root-XML localName="feed" namespaceURI="http://purl.org/atom/ns#"/> + <glob pattern="*.atom"/> + </mime-type> + + <mime-type type="application/atomcat+xml"> + <glob pattern="*.atomcat"/> + </mime-type> + <mime-type type="application/atomicmail"/> + <mime-type type="application/atomsvc+xml"> + <glob pattern="*.atomsvc"/> + </mime-type> + <mime-type type="application/auth-policy+xml"/> + <mime-type type="application/batch-smtp"/> + <mime-type type="application/beep+xml"/> + <mime-type type="application/cals-1840"/> + <mime-type type="application/ccxml+xml"> + <glob pattern="*.ccxml"/> + </mime-type> + <mime-type type="application/cea-2018+xml"/> + <mime-type type="application/cellml+xml"/> + <mime-type type="application/cnrp+xml"/> + <mime-type type="application/commonground"/> + <mime-type type="application/conference-info+xml"/> + <mime-type type="application/cpl+xml"/> + <mime-type type="application/csta+xml"/> + <mime-type type="application/cstadata+xml"/> + <mime-type type="application/cu-seeme"> + <glob pattern="*.cu"/> + </mime-type> + <mime-type type="application/cybercash"/> + <mime-type type="application/davmount+xml"> + <glob pattern="*.davmount"/> + </mime-type> + <mime-type type="application/dca-rft"/> + <mime-type type="application/dec-dx"/> + <mime-type type="application/dialog-info+xml"/> + <mime-type type="application/dicom"/> + <mime-type type="application/dns"/> + <mime-type type="application/dvcs"/> + <mime-type type="application/ecmascript"> + <glob pattern="*.ecma"/> + </mime-type> + <mime-type type="application/edi-consent"/> + <mime-type type="application/edi-x12"/> + <mime-type type="application/edifact"/> + <mime-type type="application/emma+xml"> + <glob pattern="*.emma"/> + </mime-type> + <mime-type type="application/epp+xml"/> + + <mime-type type="application/epub+zip"> + <acronym>EPUB</acronym> + <comment>Electronic Publication</comment> + <magic priority="50"> + <match value="PK\003\004" type="string" offset="0"> + <match value="mimetypeapplication/epub+zip" type="string" offset="30"/> + </match> </magic> - <glob pattern="*.txt" /> - <glob pattern="*.asc" /> + <glob pattern="*.epub"/> + </mime-type> - <!-- TIKA-85: http://www.apache.org/dev/svn-eol-style.txt --> - <glob pattern="INSTALL" /> - <glob pattern="KEYS" /> - <glob pattern="Makefile" /> - <glob pattern="README" /> - <glob pattern="abs-linkmap" /> - <glob pattern="abs-menulinks" /> - <glob pattern="*.aart" /> - <glob pattern="*.ac" /> - <glob pattern="*.am" /> - <glob pattern="*.bat" /> - <glob pattern="*.c" /> - <glob pattern="*.cat" /> - <glob pattern="*.cgi" /> - <glob pattern="*.classpath" /> - <glob pattern="*.cmd" /> - <glob pattern="*.conf" /> - <glob pattern="*.config" /> - <glob pattern="*.cpp" /> - <glob pattern="*.css" /> - <glob pattern="*.cwiki" /> - <glob pattern="*.data" /> - <glob pattern="*.dcl" /> - <glob pattern="*.dtd" /> - <glob pattern="*.egrm" /> - <glob pattern="*.ent" /> - <glob pattern="*.ft" /> - <glob pattern="*.fn" /> - <glob pattern="*.fv" /> - <glob pattern="*.grm" /> - <glob pattern="*.g" /> - <glob pattern="*.h" /> - <glob pattern=".htaccess" /> - <glob pattern="*.ihtml" /> - <glob pattern="*.in" /> - <glob pattern="*.java" /> - <glob pattern="*.jmx" /> - <glob pattern="*.jsp" /> - <glob pattern="*.js" /> - <glob pattern="*.junit" /> - <glob pattern="*.jx" /> - <glob pattern="*.manifest" /> - <glob pattern="*.m4" /> - <glob pattern="*.mf" /> - <glob pattern="*.MF" /> - <glob pattern="*.meta" /> - <glob pattern="*.n3" /> - <glob pattern="*.pen" /> - <glob pattern="*.pl" /> - <glob pattern="*.pm" /> - <glob pattern="*.pod" /> - <glob pattern="*.pom" /> - <glob pattern="*.project" /> - <glob pattern="*.properties" /> - <glob pattern="*.py" /> - <glob pattern="*.rb" /> - <glob pattern="*.rdf" /> - <glob pattern="*.rnc" /> - <glob pattern="*.rng" /> - <glob pattern="*.rnx" /> - <glob pattern="*.roles" /> - <glob pattern="*.sh" /> - <glob pattern="*.sql" /> - <glob pattern="*.tld" /> - <glob pattern="*.types" /> - <glob pattern="*.vm" /> - <glob pattern="*.vsl" /> - <glob pattern="*.wsdd" /> - <glob pattern="*.wsdl" /> - <glob pattern="*.xargs" /> - <glob pattern="*.xcat" /> - <glob pattern="*.xconf" /> - <glob pattern="*.xegrm" /> - <glob pattern="*.xgrm" /> - <glob pattern="*.xlex" /> - <glob pattern="*.xlog" /> - <glob pattern="*.xmap" /> - <glob pattern="*.xroles" /> - <glob pattern="*.xsamples" /> - <glob pattern="*.xsp" /> - <glob pattern="*.xul" /> - <glob pattern="*.xweb" /> - <glob pattern="*.xwelcome" /> + <mime-type type="application/eshop"/> + <mime-type type="application/example"/> + <mime-type type="application/fastinfoset"/> + <mime-type type="application/fastsoap"/> + <mime-type type="application/fits"/> + <mime-type type="application/font-tdpfr"> + <glob pattern="*.pfr"/> + </mime-type> + <mime-type type="application/h224"/> + <mime-type type="application/http"/> + <mime-type type="application/hyperstudio"> + <glob pattern="*.stk"/> + </mime-type> + <mime-type type="application/ibe-key-request+xml"/> + <mime-type type="application/ibe-pkg-reply+xml"/> + <mime-type type="application/ibe-pp-data"/> + <mime-type type="application/iges"/> + <mime-type type="application/im-iscomposing+xml"/> + <mime-type type="application/index"/> + <mime-type type="application/index.cmd"/> + <mime-type type="application/index.obj"/> + <mime-type type="application/index.response"/> + <mime-type type="application/index.vnd"/> + <mime-type type="application/iotp"/> + <mime-type type="application/ipp"/> + <mime-type type="application/isup"/> + + <mime-type type="application/java-archive"> + <sub-class-of type="application/zip"/> + <glob pattern="*.jar"/> </mime-type> - <mime-type type="text/html"> + <mime-type type="application/java-serialized-object"> + <glob pattern="*.ser"/> + </mime-type> + + <mime-type type="application/javascript"> + <sub-class-of type="text/plain"/> + <glob pattern="*.js"/> + </mime-type> + + <mime-type type="application/json"> + <sub-class-of type="application/javascript"/> + <glob pattern="*.json"/> + </mime-type> + + <mime-type type="application/java-vm"> + <magic priority="40"> + <match value="0xcafebabe" type="string" offset="0" /> + </magic> + <glob pattern="*.class"/> + </mime-type> + + <mime-type type="application/kpml-request+xml"/> + <mime-type type="application/kpml-response+xml"/> + <mime-type type="application/lost+xml"> + <glob pattern="*.lostxml"/> + </mime-type> + + <mime-type type="application/mac-binhex40"> + <alias type="application/mac-binhex"/> + <alias type="application/binhex"/> <magic priority="50"> - <match value="&lt;!DOCTYPE HTML" type="string" offset="0:64" /> - <match value="&lt;!doctype html" type="string" offset="0:64" /> - <match value="&lt;HEAD" type="string" offset="0:64" /> - <match value="&lt;head" type="string" offset="0:64" /> - <match value="&lt;TITLE" type="string" offset="0:64" /> - <match value="&lt;title" type="string" offset="0:64" /> - <match value="&lt;html" type="string" offset="0:64" /> - <match value="&lt;HTML" type="string" offset="0:64" /> - <match value="&lt;BODY" type="string" offset="0" /> - <match value="&lt;body" type="string" offset="0" /> - <match value="&lt;TITLE" type="string" offset="0" /> - <match value="&lt;title" type="string" offset="0" /> - <match value="&lt;!--" type="string" offset="0" /> - <match value="&lt;h1" type="string" offset="0" /> - <match value="&lt;H1" type="string" offset="0" /> - <match value="&lt;!doctype HTML" type="string" offset="0" /> - <match value="&lt;!DOCTYPE html" type="string" offset="0" /> + <match value="must\ be\ converted\ with\ BinHex" type="string" offset="11"/> </magic> - <glob pattern="*.html" /> - <glob pattern="*.htm" /> + <glob pattern="*.hqx"/> </mime-type> - <mime-type type="application/xhtml+xml"> - <sub-class-of type="application/xml" /> - <glob pattern="*.xhtml" /> - <glob pattern="*.xht" /> - <root-XML namespaceURI="http://www.w3.org/1999/xhtml" localName="html" /> + <mime-type type="application/mac-compactpro"> + <glob pattern="*.cpt"/> + </mime-type> + + <mime-type type="application/macwriteii"/> + <mime-type type="application/marc"> + <glob pattern="*.mrc"/> + </mime-type> + <mime-type type="application/mathematica"> + <glob pattern="*.ma"/> + <glob pattern="*.nb"/> + <glob pattern="*.mb"/> + </mime-type> + <mime-type type="application/mathml+xml"> + <glob pattern="*.mathml"/> + </mime-type> + <mime-type type="application/mbms-associated-procedure-description+xml"/> + <mime-type type="application/mbms-deregister+xml"/> + <mime-type type="application/mbms-envelope+xml"/> + <mime-type type="application/mbms-msk+xml"/> + <mime-type type="application/mbms-msk-response+xml"/> + <mime-type type="application/mbms-protection-description+xml"/> + <mime-type type="application/mbms-reception-report+xml"/> + <mime-type type="application/mbms-register+xml"/> + <mime-type type="application/mbms-register-response+xml"/> + <mime-type type="application/mbms-user-service-description+xml"/> + <mime-type type="application/mbox"> + <sub-class-of type="text/plain"/> + <glob pattern="*.mbox"/> + </mime-type> + <mime-type type="application/media_control+xml"/> + <mime-type type="application/mediaservercontrol+xml"> + <glob pattern="*.mscml"/> + </mime-type> + <mime-type type="application/mikey"/> + <mime-type type="application/moss-keys"/> + <mime-type type="application/moss-signature"/> + <mime-type type="application/mosskey-data"/> + <mime-type type="application/mosskey-request"/> + <mime-type type="application/mp4"> + <glob pattern="*.mp4s"/> + </mime-type> + <mime-type type="application/mpeg4-generic"/> + <mime-type type="application/mpeg4-iod"/> + <mime-type type="application/mpeg4-iod-xmt"/> + + <!-- http://www.iana.org/assignments/media-types/application/msword --> + <mime-type type="application/msword"> + <alias type="application/vnd.ms-word"/> + <comment>Microsoft Word Document</comment> + <magic priority="50"> + <match value="Microsoft\ Word\ 6.0\ Document" type="string" offset="2080"/> + <match value="Documento\ Microsoft\ Word\ 6" type="string" offset="2080"/> + <match value="MSWordDoc" type="string" offset="2112"/> + <match value="0x31be0000" type="big32" offset="0"/> + <match value="PO^Q`" type="string" offset="0"/> + <match value="\376\067\0\043" type="string" offset="0"/> + <match value="\333\245-\0\0\0" type="string" offset="0"/> + <match value="\354\245\301" type="string" offset="512"/> + <match value="\320\317\021\340\241\261\032\341" type="string" offset="0"/> + <match value="\224\246\056" type="string" offset="0"/> + <match value="R\0o\0o\0t\0\ \0E\0n\0t\0r\0y" type="string" offset="512"/> + </magic> + <glob pattern="*.doc"/> + <glob pattern="*.dot"/> + <sub-class-of type="application/x-tika-msoffice"/> + </mime-type> + + <mime-type type="application/mxf"> + <glob pattern="*.mxf"/> + </mime-type> + <mime-type type="application/nasdata"/> + <mime-type type="application/news-checkgroups"/> + <mime-type type="application/news-groupinfo"/> + <mime-type type="application/news-transmission"/> + <mime-type type="application/nss"/> + <mime-type type="application/ocsp-request"/> + <mime-type type="application/ocsp-response"/> + + <mime-type type="application/octet-stream"> + <magic priority="50"> + <match value="#\ This\ is\ a\ shell\ archive" type="string" offset="10"/> + <match value="\037\036" type="string" offset="0"/> + <match value="017437" type="host16" offset="0"/> + <match value="0x1fff" type="host16" offset="0"/> + <match value="\377\037" type="string" offset="0"/> + <match value="0145405" type="host16" offset="0"/> + </magic> + <glob pattern="*.bin"/> + <glob pattern="*.dms"/> + <glob pattern="*.lha"/> + <glob pattern="*.lrf"/> + <glob pattern="*.lzh"/> + <glob pattern="*.so"/> + <glob pattern="*.iso"/> + <glob pattern="*.dmg"/> + <glob pattern="*.dist"/> + <glob pattern="*.distz"/> + <glob pattern="*.pkg"/> + <glob pattern="*.bpk"/> + <glob pattern="*.dump"/> + <glob pattern="*.elc"/> + <glob pattern="*.deploy"/> + </mime-type> + + <mime-type type="application/oda"> + <glob pattern="*.oda"/> + </mime-type> + <mime-type type="application/oebps-package+xml"> + <glob pattern="*.opf"/> + </mime-type> + + <mime-type type="application/ogg"> + <alias type="application/x-ogg"/> + <magic priority="50"> + <match value="OggS" type="string" offset="0"/> + </magic> + <glob pattern="*.ogx"/> + </mime-type> + + <mime-type type="application/onenote"> + <glob pattern="*.onetoc"/> + <glob pattern="*.onetoc2"/> + <glob pattern="*.onetmp"/> + <glob pattern="*.onepkg"/> + </mime-type> + <mime-type type="application/parityfec"/> + <mime-type type="application/patch-ops-error+xml"> + <glob pattern="*.xer"/> + </mime-type> + + <mime-type type="application/pdf"> + <alias type="application/x-pdf"/> + <acronym>PDF</acronym> + <comment>Portable Document Format</comment> + <magic priority="50"> + <match value="%PDF-" type="string" offset="0"/> + </magic> + <glob pattern="*.pdf"/> + </mime-type> + + <mime-type type="application/pgp-encrypted"> + <glob pattern="*.pgp"/> + </mime-type> + <mime-type type="application/pgp-keys"/> + <mime-type type="application/pgp-signature"> + <glob pattern="*.asc"/> + <glob pattern="*.sig"/> + </mime-type> + <mime-type type="application/pics-rules"> + <glob pattern="*.prf"/> + </mime-type> + <mime-type type="application/pidf+xml"/> + <mime-type type="application/pidf-diff+xml"/> + <mime-type type="application/pkcs10"> + <glob pattern="*.p10"/> + </mime-type> + <mime-type type="application/pkcs7-mime"> + <glob pattern="*.p7m"/> + <glob pattern="*.p7c"/> + </mime-type> + <mime-type type="application/pkcs7-signature"> + <glob pattern="*.p7s"/> + </mime-type> + <mime-type type="application/pkix-cert"> + <glob pattern="*.cer"/> + </mime-type> + <mime-type type="application/pkix-crl"> + <glob pattern="*.crl"/> + </mime-type> + <mime-type type="application/pkix-pkipath"> + <glob pattern="*.pkipath"/> + </mime-type> + <mime-type type="application/pkixcmp"> + <glob pattern="*.pki"/> + </mime-type> + <mime-type type="application/pls+xml"> + <glob pattern="*.pls"/> + </mime-type> + <mime-type type="application/poc-settings+xml"/> + + <mime-type type="application/postscript"> + <comment>PostScript</comment> + <magic priority="50"> + <match value="%!" type="string" offset="0" /> + <match value="\004%!" type="string" offset="0" /> + <!-- Windows format EPS --> + <match value="0xc5d0d3c6" type="string" offset="0"/> + </magic> + <glob pattern="*.ai"/> + <glob pattern="*.ps"/> + <glob pattern="*.eps"/> + <glob pattern="*.epsf"/> + <glob pattern="*.epsi"/> + </mime-type> + + <mime-type type="application/prs.alvestrand.titrax-sheet"/> + <mime-type type="application/prs.cww"> + <glob pattern="*.cww"/> + </mime-type> + <mime-type type="application/prs.nprend"/> + <mime-type type="application/prs.plucker"/> + <mime-type type="application/qsig"/> + + <mime-type type="application/rdf+xml"> + <root-XML localName="RDF"/> + <root-XML localName="RDF" namespaceURI="http://www.w3.org/1999/02/22-rdf-syntax-ns#"/> + <sub-class-of type="application/xml"/> + <acronym>RDF/XML</acronym> + <comment>XML syntax for RDF graphs</comment> + <glob pattern="*.rdf"/> + <glob pattern="*.owl"/> + <glob pattern="^rdf$" isregex="true"/> + <glob pattern="^owl$" isregex="true"/> + </mime-type> + + <mime-type type="application/reginfo+xml"> + <glob pattern="*.rif"/> + </mime-type> + <mime-type type="application/relax-ng-compact-syntax"> + <sub-class-of type="text/plain"/> + <glob pattern="*.rnc"/> + </mime-type> + <mime-type type="application/remote-printing"/> + <mime-type type="application/resource-lists+xml"> + <glob pattern="*.rl"/> + </mime-type> + <mime-type type="application/resource-lists-diff+xml"> + <glob pattern="*.rld"/> + </mime-type> + <mime-type type="application/riscos"/> + <mime-type type="application/rlmi+xml"/> + <mime-type type="application/rls-services+xml"> + <glob pattern="*.rs"/> + </mime-type> + <mime-type type="application/rsd+xml"> + <glob pattern="*.rsd"/> + </mime-type> + + <mime-type type="application/rss+xml"> + <alias type="text/rss"/> + <root-XML localName="rss"/> + <root-XML namespaceURI="http://purl.org/rss/1.0/"/> + <glob pattern="*.rss"/> + </mime-type> + + <mime-type type="application/rtf"> + <alias type="text/rtf"/> + <magic priority="50"> + <match value="{\\rtf" type="string" offset="0"/> + </magic> + <glob pattern="*.rtf"/> + <sub-class-of type="text/plain"/> + </mime-type> + + <mime-type type="application/rtx"/> + <mime-type type="application/samlassertion+xml"/> + <mime-type type="application/samlmetadata+xml"/> + <mime-type type="application/sbml+xml"> + <glob pattern="*.sbml"/> + </mime-type> + <mime-type type="application/scvp-cv-request"> + <glob pattern="*.scq"/> + </mime-type> + <mime-type type="application/scvp-cv-response"> + <glob pattern="*.scs"/> + </mime-type> + <mime-type type="application/scvp-vp-request"> + <glob pattern="*.spq"/> + </mime-type> + <mime-type type="application/scvp-vp-response"> + <glob pattern="*.spp"/> + </mime-type> + <mime-type type="application/sdp"> + <glob pattern="*.sdp"/> + </mime-type> + <mime-type type="application/set-payment"/> + <mime-type type="application/set-payment-initiation"> + <glob pattern="*.setpay"/> + </mime-type> + <mime-type type="application/set-registration"/> + <mime-type type="application/set-registration-initiation"> + <glob pattern="*.setreg"/> + </mime-type> + <mime-type type="application/sgml"/> + <mime-type type="application/sgml-open-catalog"/> + <mime-type type="application/shf+xml"> + <glob pattern="*.shf"/> + </mime-type> + <mime-type type="application/sieve"/> + <mime-type type="application/simple-filter+xml"/> + <mime-type type="application/simple-message-summary"/> + <mime-type type="application/simplesymbolcontainer"/> + <mime-type type="application/slate"/> + <mime-type type="application/smil"/> + <mime-type type="application/smil+xml"> + <glob pattern="*.smi"/> + <glob pattern="*.smil"/> + </mime-type> + <mime-type type="application/soap+fastinfoset"/> + <mime-type type="application/soap+xml"/> + <mime-type type="application/sparql-query"> + <glob pattern="*.rq"/> + </mime-type> + <mime-type type="application/sparql-results+xml"> + <glob pattern="*.srx"/> + </mime-type> + <mime-type type="application/spirits-event+xml"/> + <mime-type type="application/srgs"> + <glob pattern="*.gram"/> + </mime-type> + <mime-type type="application/srgs+xml"> + <glob pattern="*.grxml"/> + </mime-type> + <mime-type type="application/ssml+xml"> + <glob pattern="*.ssml"/> + </mime-type> + <mime-type type="application/timestamp-query"/> + <mime-type type="application/timestamp-reply"/> + <mime-type type="application/tve-trigger"/> + <mime-type type="application/ulpfec"/> + <mime-type type="application/vemmi"/> + <mime-type type="application/vividence.scriptfile"/> + <mime-type type="application/vnd.3gpp.bsf+xml"/> + <mime-type type="application/vnd.3gpp.pic-bw-large"> + <glob pattern="*.plb"/> + </mime-type> + <mime-type type="application/vnd.3gpp.pic-bw-small"> + <glob pattern="*.psb"/> + </mime-type> + <mime-type type="application/vnd.3gpp.pic-bw-var"> + <glob pattern="*.pvb"/> + </mime-type> + <mime-type type="application/vnd.3gpp.sms"/> + <mime-type type="application/vnd.3gpp2.bcmcsinfo+xml"/> + <mime-type type="application/vnd.3gpp2.sms"/> + <mime-type type="application/vnd.3gpp2.tcap"> + <glob pattern="*.tcap"/> + </mime-type> + <mime-type type="application/vnd.3m.post-it-notes"> + <glob pattern="*.pwn"/> + </mime-type> + <mime-type type="application/vnd.accpac.simply.aso"> + <glob pattern="*.aso"/> + </mime-type> + <mime-type type="application/vnd.accpac.simply.imp"> + <glob pattern="*.imp"/> + </mime-type> + <mime-type type="application/vnd.acucobol"> + <glob pattern="*.acu"/> + </mime-type> + <mime-type type="application/vnd.acucorp"> + <glob pattern="*.atc"/> + <glob pattern="*.acutc"/> + </mime-type> + <mime-type type="application/vnd.adobe.air-application-installer-package+zip"> + <glob pattern="*.air"/> + </mime-type> + <mime-type type="application/vnd.adobe.xdp+xml"> + <glob pattern="*.xdp"/> + </mime-type> + <mime-type type="application/vnd.adobe.xfdf"> + <glob pattern="*.xfdf"/> + </mime-type> + <mime-type type="application/vnd.aether.imp"/> + <mime-type type="application/vnd.airzip.filesecure.azf"> + <glob pattern="*.azf"/> + </mime-type> + <mime-type type="application/vnd.airzip.filesecure.azs"> + <glob pattern="*.azs"/> + </mime-type> + <mime-type type="application/vnd.amazon.ebook"> + <glob pattern="*.azw"/> + </mime-type> + <mime-type type="application/vnd.americandynamics.acc"> + <glob pattern="*.acc"/> + </mime-type> + <mime-type type="application/vnd.amiga.ami"> + <glob pattern="*.ami"/> + </mime-type> + <mime-type type="application/vnd.android.package-archive"> + <glob pattern="*.apk"/> + </mime-type> + <mime-type type="application/vnd.anser-web-certificate-issue-initiation"> + <glob pattern="*.cii"/> + </mime-type> + <mime-type type="application/vnd.anser-web-funds-transfer-initiation"> + <glob pattern="*.fti"/> + </mime-type> + <mime-type type="application/vnd.antix.game-component"> + <glob pattern="*.atx"/> + </mime-type> + <mime-type type="application/vnd.apple.installer+xml"> + <glob pattern="*.mpkg"/> + </mime-type> + <mime-type type="application/vnd.arastra.swi"> + <glob pattern="*.swi"/> + </mime-type> + <mime-type type="application/vnd.audiograph"> + <glob pattern="*.aep"/> + </mime-type> + <mime-type type="application/vnd.autopackage"/> + <mime-type type="application/vnd.avistar+xml"/> + <mime-type type="application/vnd.blueice.multipass"> + <glob pattern="*.mpm"/> + </mime-type> + <mime-type type="application/vnd.bluetooth.ep.oob"/> + <mime-type type="application/vnd.bmi"> + <glob pattern="*.bmi"/> + </mime-type> + <mime-type type="application/vnd.businessobjects"> + <glob pattern="*.rep"/> + </mime-type> + <mime-type type="application/vnd.cab-jscript"/> + <mime-type type="application/vnd.canon-cpdl"/> + <mime-type type="application/vnd.canon-lips"/> + <mime-type type="application/vnd.cendio.thinlinc.clientconf"/> + <mime-type type="application/vnd.chemdraw+xml"> + <glob pattern="*.cdxml"/> + </mime-type> + <mime-type type="application/vnd.chipnuts.karaoke-mmd"> + <glob pattern="*.mmd"/> + </mime-type> + <mime-type type="application/vnd.cinderella"> + <glob pattern="*.cdy"/> + </mime-type> + <mime-type type="application/vnd.cirpack.isdn-ext"/> + <mime-type type="application/vnd.claymore"> + <glob pattern="*.cla"/> + </mime-type> + <mime-type type="application/vnd.clonk.c4group"> + <glob pattern="*.c4g"/> + <glob pattern="*.c4d"/> + <glob pattern="*.c4f"/> + <glob pattern="*.c4p"/> + <glob pattern="*.c4u"/> + </mime-type> + <mime-type type="application/vnd.commerce-battelle"/> + <mime-type type="application/vnd.commonspace"> + <glob pattern="*.csp"/> + </mime-type> + <mime-type type="application/vnd.contact.cmsg"> + <glob pattern="*.cdbcmsg"/> + </mime-type> + <mime-type type="application/vnd.cosmocaller"> + <glob pattern="*.cmc"/> + </mime-type> + <mime-type type="application/vnd.crick.clicker"> + <glob pattern="*.clkx"/> + </mime-type> + <mime-type type="application/vnd.crick.clicker.keyboard"> + <glob pattern="*.clkk"/> + </mime-type> + <mime-type type="application/vnd.crick.clicker.palette"> + <glob pattern="*.clkp"/> + </mime-type> + <mime-type type="application/vnd.crick.clicker.template"> + <glob pattern="*.clkt"/> + </mime-type> + <mime-type type="application/vnd.crick.clicker.wordbank"> + <glob pattern="*.clkw"/> + </mime-type> + <mime-type type="application/vnd.criticaltools.wbs+xml"> + <glob pattern="*.wbs"/> + </mime-type> + <mime-type type="application/vnd.ctc-posml"> + <glob pattern="*.pml"/> + </mime-type> + <mime-type type="application/vnd.ctct.ws+xml"/> + <mime-type type="application/vnd.cups-pdf"/> + <mime-type type="application/vnd.cups-postscript"/> + <mime-type type="application/vnd.cups-ppd"> + <glob pattern="*.ppd"/> + </mime-type> + <mime-type type="application/vnd.cups-raster"/> + <mime-type type="application/vnd.cups-raw"/> + <mime-type type="application/vnd.curl.car"> + <glob pattern="*.car"/> + </mime-type> + <mime-type type="application/vnd.curl.pcurl"> + <glob pattern="*.pcurl"/> + </mime-type> + <mime-type type="application/vnd.cybank"/> + <mime-type type="application/vnd.data-vision.rdz"> + <glob pattern="*.rdz"/> + </mime-type> + <mime-type type="application/vnd.denovo.fcselayout-link"> + <glob pattern="*.fe_launch"/> + </mime-type> + <mime-type type="application/vnd.dir-bi.plate-dl-nosuffix"/> + <mime-type type="application/vnd.dna"> + <glob pattern="*.dna"/> + </mime-type> + <mime-type type="application/vnd.dolby.mlp"> + <glob pattern="*.mlp"/> + </mime-type> + <mime-type type="application/vnd.dolby.mobile.1"/> + <mime-type type="application/vnd.dolby.mobile.2"/> + <mime-type type="application/vnd.dpgraph"> + <glob pattern="*.dpg"/> + </mime-type> + <mime-type type="application/vnd.dreamfactory"> + <glob pattern="*.dfac"/> + </mime-type> + <mime-type type="application/vnd.dvb.esgcontainer"/> + <mime-type type="application/vnd.dvb.ipdcdftnotifaccess"/> + <mime-type type="application/vnd.dvb.ipdcesgaccess"/> + <mime-type type="application/vnd.dvb.ipdcroaming"/> + <mime-type type="application/vnd.dvb.iptv.alfec-base"/> + <mime-type type="application/vnd.dvb.iptv.alfec-enhancement"/> + <mime-type type="application/vnd.dvb.notif-aggregate-root+xml"/> + <mime-type type="application/vnd.dvb.notif-container+xml"/> + <mime-type type="application/vnd.dvb.notif-generic+xml"/> + <mime-type type="application/vnd.dvb.notif-ia-msglist+xml"/> + <mime-type type="application/vnd.dvb.notif-ia-registration-request+xml"/> + <mime-type type="application/vnd.dvb.notif-ia-registration-response+xml"/> + <mime-type type="application/vnd.dvb.notif-init+xml"/> + <mime-type type="application/vnd.dxr"/> + <mime-type type="application/vnd.dynageo"> + <glob pattern="*.geo"/> + </mime-type> + <mime-type type="application/vnd.ecdis-update"/> + <mime-type type="application/vnd.ecowin.chart"> + <glob pattern="*.mag"/> + </mime-type> + <mime-type type="application/vnd.ecowin.filerequest"/> + <mime-type type="application/vnd.ecowin.fileupdate"/> + <mime-type type="application/vnd.ecowin.series"/> + <mime-type type="application/vnd.ecowin.seriesrequest"/> + <mime-type type="application/vnd.ecowin.seriesupdate"/> + <mime-type type="application/vnd.emclient.accessrequest+xml"/> + <mime-type type="application/vnd.enliven"> + <glob pattern="*.nml"/> + </mime-type> + <mime-type type="application/vnd.epson.esf"> + <glob pattern="*.esf"/> + </mime-type> + <mime-type type="application/vnd.epson.msf"> + <glob pattern="*.msf"/> + </mime-type> + <mime-type type="application/vnd.epson.quickanime"> + <glob pattern="*.qam"/> + </mime-type> + <mime-type type="application/vnd.epson.salt"> + <glob pattern="*.slt"/> + </mime-type> + <mime-type type="application/vnd.epson.ssf"> + <glob pattern="*.ssf"/> + </mime-type> + <mime-type type="application/vnd.ericsson.quickcall"/> + <mime-type type="application/vnd.eszigno3+xml"> + <glob pattern="*.es3"/> + <glob pattern="*.et3"/> + </mime-type> + <mime-type type="application/vnd.etsi.aoc+xml"/> + <mime-type type="application/vnd.etsi.cug+xml"/> + <mime-type type="application/vnd.etsi.iptvcommand+xml"/> + <mime-type type="application/vnd.etsi.iptvdiscovery+xml"/> + <mime-type type="application/vnd.etsi.iptvprofile+xml"/> + <mime-type type="application/vnd.etsi.iptvsad-bc+xml"/> + <mime-type type="application/vnd.etsi.iptvsad-cod+xml"/> + <mime-type type="application/vnd.etsi.iptvsad-npvr+xml"/> + <mime-type type="application/vnd.etsi.iptvueprofile+xml"/> + <mime-type type="application/vnd.etsi.mcid+xml"/> + <mime-type type="application/vnd.etsi.sci+xml"/> + <mime-type type="application/vnd.etsi.simservs+xml"/> + <mime-type type="application/vnd.eudora.data"/> + <mime-type type="application/vnd.ezpix-album"> + <glob pattern="*.ez2"/> + </mime-type> + <mime-type type="application/vnd.ezpix-package"> + <glob pattern="*.ez3"/> + </mime-type> + <mime-type type="application/vnd.f-secure.mobile"/> + <mime-type type="application/vnd.fdf"> + <glob pattern="*.fdf"/> + </mime-type> + <mime-type type="application/vnd.fdsn.mseed"> + <glob pattern="*.mseed"/> + </mime-type> + <mime-type type="application/vnd.fdsn.seed"> + <glob pattern="*.seed"/> + <glob pattern="*.dataless"/> + </mime-type> + <mime-type type="application/vnd.ffsns"/> + <mime-type type="application/vnd.fints"/> + <mime-type type="application/vnd.flographit"> + <glob pattern="*.gph"/> + </mime-type> + <mime-type type="application/vnd.fluxtime.clip"> + <glob pattern="*.ftc"/> + </mime-type> + <mime-type type="application/vnd.font-fontforge-sfd"/> + <mime-type type="application/vnd.framemaker"> + <glob pattern="*.fm"/> + <glob pattern="*.frame"/> + <glob pattern="*.maker"/> + <glob pattern="*.book"/> + </mime-type> + <mime-type type="application/vnd.frogans.fnc"> + <glob pattern="*.fnc"/> + </mime-type> + <mime-type type="application/vnd.frogans.ltf"> + <glob pattern="*.ltf"/> + </mime-type> + <mime-type type="application/vnd.fsc.weblaunch"> + <glob pattern="*.fsc"/> + </mime-type> + <mime-type type="application/vnd.fujitsu.oasys"> + <glob pattern="*.oas"/> + </mime-type> + <mime-type type="application/vnd.fujitsu.oasys2"> + <glob pattern="*.oa2"/> + </mime-type> + <mime-type type="application/vnd.fujitsu.oasys3"> + <glob pattern="*.oa3"/> + </mime-type> + <mime-type type="application/vnd.fujitsu.oasysgp"> + <glob pattern="*.fg5"/> + </mime-type> + <mime-type type="application/vnd.fujitsu.oasysprs"> + <glob pattern="*.bh2"/> + </mime-type> + <mime-type type="application/vnd.fujixerox.art-ex"/> + <mime-type type="application/vnd.fujixerox.art4"/> + <mime-type type="application/vnd.fujixerox.hbpl"/> + <mime-type type="application/vnd.fujixerox.ddd"> + <glob pattern="*.ddd"/> + </mime-type> + <mime-type type="application/vnd.fujixerox.docuworks"> + <glob pattern="*.xdw"/> + </mime-type> + <mime-type type="application/vnd.fujixerox.docuworks.binder"> + <glob pattern="*.xbd"/> + </mime-type> + <mime-type type="application/vnd.fut-misnet"/> + <mime-type type="application/vnd.fuzzysheet"> + <glob pattern="*.fzs"/> + </mime-type> + <mime-type type="application/vnd.genomatix.tuxedo"> + <glob pattern="*.txd"/> + </mime-type> + <mime-type type="application/vnd.geogebra.file"> + <glob pattern="*.ggb"/> + </mime-type> + <mime-type type="application/vnd.geogebra.tool"> + <glob pattern="*.ggt"/> + </mime-type> + <mime-type type="application/vnd.geometry-explorer"> + <glob pattern="*.gex"/> + <glob pattern="*.gre"/> + </mime-type> + <mime-type type="application/vnd.gmx"> + <glob pattern="*.gmx"/> + </mime-type> + <mime-type type="application/vnd.google-earth.kml+xml"> + <glob pattern="*.kml"/> + </mime-type> + <mime-type type="application/vnd.google-earth.kmz"> + <glob pattern="*.kmz"/> + </mime-type> + <mime-type type="application/vnd.grafeq"> + <glob pattern="*.gqf"/> + <glob pattern="*.gqs"/> + </mime-type> + <mime-type type="application/vnd.gridmp"/> + <mime-type type="application/vnd.groove-account"> + <glob pattern="*.gac"/> + </mime-type> + <mime-type type="application/vnd.groove-help"> + <glob pattern="*.ghf"/> + </mime-type> + <mime-type type="application/vnd.groove-identity-message"> + <glob pattern="*.gim"/> + </mime-type> + <mime-type type="application/vnd.groove-injector"> + <glob pattern="*.grv"/> + </mime-type> + <mime-type type="application/vnd.groove-tool-message"> + <glob pattern="*.gtm"/> + </mime-type> + <mime-type type="application/vnd.groove-tool-template"> + <glob pattern="*.tpl"/> + </mime-type> + <mime-type type="application/vnd.groove-vcard"> + <glob pattern="*.vcg"/> + </mime-type> + <mime-type type="application/vnd.handheld-entertainment+xml"> + <glob pattern="*.zmm"/> + </mime-type> + <mime-type type="application/vnd.hbci"> + <glob pattern="*.hbci"/> + </mime-type> + <mime-type type="application/vnd.hcl-bireports"/> + <mime-type type="application/vnd.hhe.lesson-player"> + <glob pattern="*.les"/> + </mime-type> + <mime-type type="application/vnd.hp-hpgl"> + <glob pattern="*.hpgl"/> + </mime-type> + <mime-type type="application/vnd.hp-hpid"> + <glob pattern="*.hpid"/> + </mime-type> + <mime-type type="application/vnd.hp-hps"> + <glob pattern="*.hps"/> + </mime-type> + <mime-type type="application/vnd.hp-jlyt"> + <glob pattern="*.jlt"/> + </mime-type> + <mime-type type="application/vnd.hp-pcl"> + <glob pattern="*.pcl"/> + </mime-type> + <mime-type type="application/vnd.hp-pclxl"> + <glob pattern="*.pclxl"/> + </mime-type> + <mime-type type="application/vnd.httphone"/> + <mime-type type="application/vnd.hydrostatix.sof-data"> + <glob pattern="*.sfd-hdstx"/> + </mime-type> + <mime-type type="application/vnd.hzn-3d-crossword"> + <glob pattern="*.x3d"/> + </mime-type> + <mime-type type="application/vnd.ibm.afplinedata"/> + <mime-type type="application/vnd.ibm.electronic-media"/> + <mime-type type="application/vnd.ibm.minipay"> + <glob pattern="*.mpy"/> + </mime-type> + <mime-type type="application/vnd.ibm.modcap"> + <glob pattern="*.afp"/> + <glob pattern="*.listafp"/> + <glob pattern="*.list3820"/> + </mime-type> + <mime-type type="application/vnd.ibm.rights-management"> + <glob pattern="*.irm"/> + </mime-type> + <mime-type type="application/vnd.ibm.secure-container"> + <glob pattern="*.sc"/> + </mime-type> + <mime-type type="application/vnd.iccprofile"> + <glob pattern="*.icc"/> + <glob pattern="*.icm"/> + </mime-type> + <mime-type type="application/vnd.igloader"> + <glob pattern="*.igl"/> + </mime-type> + <mime-type type="application/vnd.immervision-ivp"> + <glob pattern="*.ivp"/> + </mime-type> + <mime-type type="application/vnd.immervision-ivu"> + <glob pattern="*.ivu"/> + </mime-type> + <mime-type type="application/vnd.informedcontrol.rms+xml"/> + <mime-type type="application/vnd.informix-visionary"/> + <mime-type type="application/vnd.intercon.formnet"> + <glob pattern="*.xpw"/> + <glob pattern="*.xpx"/> + </mime-type> + <mime-type type="application/vnd.intertrust.digibox"/> + <mime-type type="application/vnd.intertrust.nncp"/> + <mime-type type="application/vnd.intu.qbo"> + <glob pattern="*.qbo"/> + </mime-type> + <mime-type type="application/vnd.intu.qfx"> + <glob pattern="*.qfx"/> + </mime-type> + <mime-type type="application/vnd.iptc.g2.conceptitem+xml"/> + <mime-type type="application/vnd.iptc.g2.knowledgeitem+xml"/> + <mime-type type="application/vnd.iptc.g2.newsitem+xml"/> + <mime-type type="application/vnd.iptc.g2.packageitem+xml"/> + <mime-type type="application/vnd.ipunplugged.rcprofile"> + <glob pattern="*.rcprofile"/> + </mime-type> + <mime-type type="application/vnd.irepository.package+xml"> + <glob pattern="*.irp"/> + </mime-type> + <mime-type type="application/vnd.is-xpr"> + <glob pattern="*.xpr"/> + </mime-type> + <mime-type type="application/vnd.jam"> + <glob pattern="*.jam"/> + </mime-type> + <mime-type type="application/vnd.japannet-directory-service"/> + <mime-type type="application/vnd.japannet-jpnstore-wakeup"/> + <mime-type type="application/vnd.japannet-payment-wakeup"/> + <mime-type type="application/vnd.japannet-registration"/> + <mime-type type="application/vnd.japannet-registration-wakeup"/> + <mime-type type="application/vnd.japannet-setstore-wakeup"/> + <mime-type type="application/vnd.japannet-verification"/> + <mime-type type="application/vnd.japannet-verification-wakeup"/> + <mime-type type="application/vnd.jcp.javame.midlet-rms"> + <glob pattern="*.rms"/> + </mime-type> + <mime-type type="application/vnd.jisp"> + <glob pattern="*.jisp"/> + </mime-type> + <mime-type type="application/vnd.joost.joda-archive"> + <glob pattern="*.joda"/> + </mime-type> + <mime-type type="application/vnd.kahootz"> + <glob pattern="*.ktz"/> + <glob pattern="*.ktr"/> + </mime-type> + <mime-type type="application/vnd.kde.karbon"> + <glob pattern="*.karbon"/> + </mime-type> + <mime-type type="application/vnd.kde.kchart"> + <glob pattern="*.chrt"/> + </mime-type> + <mime-type type="application/vnd.kde.kformula"> + <glob pattern="*.kfo"/> + </mime-type> + <mime-type type="application/vnd.kde.kivio"> + <glob pattern="*.flw"/> + </mime-type> + <mime-type type="application/vnd.kde.kontour"> + <glob pattern="*.kon"/> + </mime-type> + <mime-type type="application/vnd.kde.kpresenter"> + <glob pattern="*.kpr"/> + <glob pattern="*.kpt"/> + </mime-type> + <mime-type type="application/vnd.kde.kspread"> + <glob pattern="*.ksp"/> + </mime-type> + <mime-type type="application/vnd.kde.kword"> + <glob pattern="*.kwd"/> + <glob pattern="*.kwt"/> + </mime-type> + <mime-type type="application/vnd.kenameaapp"> + <glob pattern="*.htke"/> + </mime-type> + <mime-type type="application/vnd.kidspiration"> + <glob pattern="*.kia"/> + </mime-type> + <mime-type type="application/vnd.kinar"> + <glob pattern="*.kne"/> + <glob pattern="*.knp"/> + </mime-type> + <mime-type type="application/vnd.koan"> + <alias type="application/x-koan"/> + <_comment>SSEYO Koan File</_comment> + <glob pattern="*.skp"/> + <glob pattern="*.skd"/> + <glob pattern="*.skt"/> + <glob pattern="*.skm"/> + </mime-type> + <mime-type type="application/vnd.kodak-descriptor"> + <glob pattern="*.sse"/> + </mime-type> + <mime-type type="application/vnd.liberty-request+xml"/> + <mime-type type="application/vnd.llamagraphics.life-balance.desktop"> + <glob pattern="*.lbd"/> + </mime-type> + <mime-type type="application/vnd.llamagraphics.life-balance.exchange+xml"> + <glob pattern="*.lbe"/> + </mime-type> + <mime-type type="application/vnd.lotus-1-2-3"> + <glob pattern="*.123"/> + </mime-type> + <mime-type type="application/vnd.lotus-approach"> + <glob pattern="*.apr"/> + </mime-type> + <mime-type type="application/vnd.lotus-freelance"> + <glob pattern="*.pre"/> + </mime-type> + <mime-type type="application/vnd.lotus-notes"> + <glob pattern="*.nsf"/> + </mime-type> + <mime-type type="application/vnd.lotus-organizer"> + <glob pattern="*.org"/> + </mime-type> + <mime-type type="application/vnd.lotus-screencam"> + <glob pattern="*.scm"/> + </mime-type> + + <mime-type type="application/vnd.lotus-wordpro"> + <magic priority="50"> + <match value="WordPro\0" type="string" offset="0" /> + <match value="WordPro\r\373" type="string" offset="0" /> + </magic> + <glob pattern="*.lwp"/> + </mime-type> + + <mime-type type="application/vnd.macports.portpkg"> + <glob pattern="*.portpkg"/> + </mime-type> + <mime-type type="application/vnd.marlin.drm.actiontoken+xml"/> + <mime-type type="application/vnd.marlin.drm.conftoken+xml"/> + <mime-type type="application/vnd.marlin.drm.license+xml"/> + <mime-type type="application/vnd.marlin.drm.mdcf"/> + <mime-type type="application/vnd.mcd"> + <glob pattern="*.mcd"/> + </mime-type> + <mime-type type="application/vnd.medcalcdata"> + <glob pattern="*.mc1"/> + </mime-type> + <mime-type type="application/vnd.mediastation.cdkey"> + <glob pattern="*.cdkey"/> + </mime-type> + <mime-type type="application/vnd.meridian-slingshot"/> + <mime-type type="application/vnd.mfer"> + <glob pattern="*.mwf"/> + </mime-type> + <mime-type type="application/vnd.mfmp"> + <glob pattern="*.mfm"/> + </mime-type> + <mime-type type="application/vnd.micrografx.flo"> + <glob pattern="*.flo"/> + </mime-type> + <mime-type type="application/vnd.micrografx.igx"> + <glob pattern="*.igx"/> + </mime-type> + + <mime-type type="application/vnd.mif"> + <comment>FrameMaker MIF document</comment> + <alias type="application/x-mif"/> + <alias type="application/x-frame"/> + <magic priority="50"> + <match value="\&lt;MakerFile" type="string" offset="0" /> + <match value="\&lt;MIFFile" type="string" offset="0" /> + <match value="\&lt;MakerDictionary" type="string" offset="0" /> + <match value="\&lt;MakerScreenFont" type="string" offset="0" /> + <match value="\&lt;MML" type="string" offset="0" /> + <match value="\&lt;Book" type="string" offset="0" /> + <match value="\&lt;Maker" type="string" offset="0" /> + </magic> + <glob pattern="*.mif"/> + </mime-type> + + <mime-type type="application/vnd.minisoft-hp3000-save"/> + <mime-type type="application/vnd.mitsubishi.misty-guard.trustweb"/> + <mime-type type="application/vnd.mobius.daf"> + <glob pattern="*.daf"/> + </mime-type> + <mime-type type="application/vnd.mobius.dis"> + <glob pattern="*.dis"/> + </mime-type> + <mime-type type="application/vnd.mobius.mbk"> + <glob pattern="*.mbk"/> + </mime-type> + <mime-type type="application/vnd.mobius.mqy"> + <glob pattern="*.mqy"/> + </mime-type> + <mime-type type="application/vnd.mobius.msl"> + <glob pattern="*.msl"/> + </mime-type> + <mime-type type="application/vnd.mobius.plc"> + <glob pattern="*.plc"/> + </mime-type> + <mime-type type="application/vnd.mobius.txf"> + <glob pattern="*.txf"/> + </mime-type> + <mime-type type="application/vnd.mophun.application"> + <glob pattern="*.mpn"/> + </mime-type> + <mime-type type="application/vnd.mophun.certificate"> + <glob pattern="*.mpc"/> + </mime-type> + <mime-type type="application/vnd.motorola.flexsuite"/> + <mime-type type="application/vnd.motorola.flexsuite.adsi"/> + <mime-type type="application/vnd.motorola.flexsuite.fis"/> + <mime-type type="application/vnd.motorola.flexsuite.gotap"/> + <mime-type type="application/vnd.motorola.flexsuite.kmr"/> + <mime-type type="application/vnd.motorola.flexsuite.ttc"/> + <mime-type type="application/vnd.motorola.flexsuite.wem"/> + <mime-type type="application/vnd.motorola.iprm"/> + <mime-type type="application/vnd.mozilla.xul+xml"> + <glob pattern="*.xul"/> + </mime-type> + <mime-type type="application/vnd.ms-artgalry"> + <glob pattern="*.cil"/> + </mime-type> + <mime-type type="application/vnd.ms-asf"/> + <mime-type type="application/vnd.ms-cab-compressed"> + <glob pattern="*.cab"/> + </mime-type> + + <!-- http://www.iana.org/assignments/media-types/application/vnd.ms-excel --> + <mime-type type="application/vnd.ms-excel"> + <alias type="application/msexcel" /> + <comment>Microsoft Excel Spreadsheet</comment> + <magic priority="50"> + <match value="Microsoft\ Excel\ 5.0\ Worksheet" type="string" offset="2080"/> + <match value="Foglio\ di\ lavoro\ Microsoft\ Exce" type="string" offset="2080"/> + <match value="Biff5" type="string" offset="2114"/> + <match value="Biff5" type="string" offset="2121"/> + <match value="\x09\x04\x06\x00\x00\x00\x10\x00" type="string" offset="0"/> + </magic> + <glob pattern="*.xls"/> + <glob pattern="*.xlm"/> + <glob pattern="*.xla"/> + <glob pattern="*.xlc"/> + <glob pattern="*.xlt"/> + <glob pattern="*.xlw"/> + <glob pattern="*.xll"/> + <glob pattern="*.xld"/> + <sub-class-of type="application/x-tika-msoffice"/> + </mime-type> + + <mime-type type="application/vnd.ms-excel.addin.macroenabled.12"> + <comment>Office Open XML Workbook Add-in (macro-enabled)</comment> + <glob pattern="*.xlam"/> + <sub-class-of type="application/x-tika-ooxml"/> + </mime-type> + + <mime-type type="application/vnd.ms-excel.sheet.macroenabled.12"> + <comment>Office Open XML Workbook (macro-enabled)</comment> + <glob pattern="*.xlsm"/> + <sub-class-of type="application/x-tika-ooxml"/> + </mime-type> + + <mime-type type="application/vnd.ms-excel.sheet.binary.macroenabled.12"> + <comment>Microsoft Excel 2007 Binary Spreadsheet</comment> + <glob pattern="*.xlsb"/> + <sub-class-of type="application/vnd.ms-excel"/> </mime-type> - <!-- ===================================================================== --> - <!-- Microsoft Office binary file formats --> - <!-- http://www.microsoft.com/interop/docs/OfficeBinaryFormats.mspx --> - <!-- ===================================================================== --> + <mime-type type="application/vnd.ms-excel.template.macroenabled.12"> + <glob pattern="*.xltm"/> + <sub-class-of type="application/x-tika-ooxml"/> + </mime-type> - <mime-type type="application/x-tika-msoffice"> - <magic> - <match value="0xd0cf11e0a1b11ae1" type="string" offset="0:8" /> - </magic> + <mime-type type="application/vnd.ms-fontobject"> + <glob pattern="*.eot"/> + </mime-type> + <mime-type type="application/vnd.ms-htmlhelp"> + <glob pattern="*.chm"/> + </mime-type> + <mime-type type="application/vnd.ms-ims"> + <glob pattern="*.ims"/> + </mime-type> + <mime-type type="application/vnd.ms-lrm"> + <glob pattern="*.lrm"/> </mime-type> - <!-- http://www.iana.org/assignments/media-types/application/vnd.visio --> - <mime-type type="application/vnd.visio"> - <comment>Microsoft Visio Diagram</comment> - <glob pattern="*.vsd" /> - <glob pattern="*.vst" /> - <glob pattern="*.vsw" /> - <glob pattern="*.vss" /> + <mime-type type="application/vnd.ms-outlook"> + <comment>Microsoft
{'source': 'Apache Software Foundation Public Mail Archives'}