错误：java.io.IOException：错误的值类：class org.apache.hadoop.io.Text不是类Myclass
我有我的mapper和reducer如下。 但我得到了一些奇怪的例外。 我无法弄清楚为什么会抛出这种exception。
public static class MyMapper implements Mapper { @Override public void map(LongWritable key, Text value, OutputCollector output, Reporter reporter) throws IOException { Text text = new Text("someText") //process output.collect(text, infoObjeject); } } public static class MyReducer implements Reducer { @Override public void reduce(Text key, Iterator values, OutputCollector output, Reporter reporter) throws IOException { String value = "xyz" //derived in some way //process output.collect(key, new Text(value)); //exception occurs at this line } } System.out.println("Starting v14 "); JobConf conf = new JobConf(RouteBuilderJob.class); conf.setJobName("xyz"); String jarLocation =ClassUtil.findContainingJar(getClass()); System.out.println("path of jar file = " + jarLocation); conf.setJarByClass(RouteBuilderJob.class); conf.setMapOutputKeyClass(Text.class); conf.setMapOutputValueClass(Info.class); conf.setOutputKeyClass(Text.class); conf.setOutputValueClass(Text.class); //am i missing something here??? conf.setMapperClass(RouteBuilderJob.RouteMapper.class); conf.setCombinerClass(RouteBuilderJob.RouteReducer.class); conf.setReducerClass(RouteBuilderJob.RouteReducer.class); conf.setInputFormat(TextInputFormat.class); conf.setOutputFormat(TextOutputFormat.class); FileInputFormat.setInputPaths(conf, new Path(args[0])); FileOutputFormat.setOutputPath(conf, new Path(args[1])); JobClient.runJob(conf); 我得到一个例外：
Error: java.io.IOException: wrong value class: class org.apache.hadoop.io.Text is not class com.xyz.mypackage.Info at org.apache.hadoop.mapred.IFile$Writer.append(IFile.java:199) at org.apache.hadoop.mapred.Task$CombineOutputCollector.collect(Task.java:1307) at com.xyz.mypackage.job.MyJob$RouteReducer.reduce(MyJob.java:156) at com.xyz.mypackage.job.MyJob$RouteReducer.reduce(MyJob.java:1)
使用Text序列化内部信息对象（实现Writable ）
@Override public void write(DataOutput out) throws IOException { Gson gson = new Gson(); String searlizedStr = gson.toJson(this); Text.writeString(out, searlizedStr); } @Override public void readFields(DataInput in) throws IOException { String s = Text.readString(in); Gson gson = new Gson(); JsonReader jsonReader = new JsonReader(new StringReader(s)); jsonReader.setLenient(true); Info info = gson.fromJson(jsonReader, Info.class); //set fields using this.somefield = info.getsomefield() }
从技术上讲，reduce的输出类型应与输入类型相同。 如果使用组合器，组合器的输出被送入减速器，则必须如此。