Nutch의 크롤링 결과값을 최신의 ElasticSearch를 이용해 사용하고자 할경우에 ivy/ivy.xml 파일에 아래 버전을 0.90.10를 변경하고
<dependency org="org.elasticsearch" name="elasticsearch" rev="0.19.4" conf="*->default"/>
ant로 빌드시 아래와 같은 에러가 나는 경우가 있습니다.
resolve-default: [ivy:resolve] :: Ivy 2.2.0 - 20100923230623 :: http://ant.apache.org/ivy/ :: [ivy:resolve] :: loading settings :: file = /opt/apache-nutch-2.2.1/ivy/ivysettings.xml [taskdef] Could not load definitions from resource org/sonar/ant/antlib.xml. It could not be found. copy-libs: compile-core: [javac] Compiling 180 source files to /opt/apache-nutch-2.2.1/build/classes [javac] warning: [options] bootstrap class path not set in conjunction with -source 1.6 [javac] /opt/apache-nutch-2.2.1/src/java/org/apache/nutch/indexer/elastic/ElasticWriter.java:104: error: cannot find symbol [javac] if (item.failed()) { [javac] ^ [javac] symbol: method failed() [javac] location: variable item of type BulkItemResponse [javac] 1 error [javac] 1 warning BUILD FAILED /opt/apache-nutch-2.2.1/build.xml:101: Compile failed; see the compiler error output for details.
문제 되는 파일 “src/java/org/apache/nutch/indexer/elastic/ElasticWriter.java”의 104 번 라인에 있는”if (item.failed())”을 “if (item.isFailed())”로 수정 후 ant 재빌드 하면 됩니다.
– before
private void processExecute(boolean createNewBulk) { if (execute != null) { // wait for previous to finish long beforeWait = System.currentTimeMillis(); BulkResponse actionGet = execute.actionGet(); if (actionGet.hasFailures()) { for (BulkItemResponse item : actionGet) { if (item.failed()) { throw new RuntimeException("First failure in bulk: " + item.getFailureMessage()); } } }
– After
private void processExecute(boolean createNewBulk) { if (execute != null) { // wait for previous to finish long beforeWait = System.currentTimeMillis(); BulkResponse actionGet = execute.actionGet(); if (actionGet.hasFailures()) { for (BulkItemResponse item : actionGet) { if (item.isFailed()) { throw new RuntimeException("First failure in bulk: " + item.getFailureMessage()); } } }