Hadoop：LongWritable无法强制转换为org.apache.hadoop.io.IntWritable

我想取一个输入文件中给出的温度的平均值，我的Mapper和Reducer synatax似乎对我好，但我仍然收到以下错误：

Unable to load realm info from SCDynamicStore 13/02/17 08:03:28 INFO mapred.JobClient: Task Id : attempt_201302170552_0009_m_000000_1, Status : FAILED java.lang.ClassCastException: org.apache.hadoop.io.LongWritable cannot be cast to org.apache.hadoop.io.IntWritable at org.apache.hadoop.examples.TempMeasurement$TempMapper.map(TempMeasurement.java:26) at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370) at org.apache.hadoop.mapred.Child$4.run(Child.java:255) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121) at org.apache.hadoop.mapred.Child.main(Child.java:249) 

我的Mapperfunction是这样的：

 public static class TempMapper extends Mapper{ @Override protected void map(IntWritable key, Text value, Context context) throws IOException, InterruptedException { //code for getting date and temperature String temp = columns.get(3); context.write(new IntWritable(year), new FloatWritable(Float.valueOf(temp))); } } 

而减少是：

  public static class IntSumReducer extends Reducer { private FloatWritable result = new FloatWritable(); public void reduce(IntWritable key, Iterable values, Context context ) throws IOException, InterruptedException { //code for making calculations context.write(key, result); } } 

输入文件如下：

 11111 , 0,19900101, 44.04 , 11112, 0, 19900102, 50.00, 11113, 3, 19910203, 30.00, 

任何帮助，将不胜感激

Answer 1

映射文本文件的映射器的键类始终是LongWritable 。 那是因为它包含当前行的字节偏移量，这很容易溢出整数。

基本上你需要将代码更改为：

 public static class TempMapper extends Mapper{ @Override protected void map(LongWritable key, Text value, Context context) throws IOException, InterruptedException { //code for getting date and temperature String temp = columns.get(3); context.write(new IntWritable(year), new FloatWritable(Float.valueOf(temp))); } }