Java Hadoop Mapper如何发送多个值

我的映射器需要发送以下元组：

我希望将减少发送者作为密钥发送给reducer，并将prodID和速率作为值一起发送，因为它们是减少阶段所需的。 这是最好的方法吗？

 public void map(Object key, Text value, Context context) throws IOException, InterruptedException { String[] col = value.toString().split(","); custID.set(col[0]); data.set(col[1] + "," + col[2]); context.write(custID, data); } public void reduce(Text key, Iterable values, Context context) throws IOException, InterruptedException { for (Text val : values) { String[] temp = val.toString().split(","); Text rate = new Text(temp[1]); result.set(rate); context.write(key, result); } } 

Answer 1

我能想到的最简单的方法就是将它们合并为一个字符串：

 output.collect(custID, prodID + "," + rate); 

然后，如果在减速器上备份则拆分。

如果你从mapper中发布更多代码，我们可以举一个更好的例子。

更新：那就是说，你问了最好的方法。 最正确的方法可能是创建一个单独的类分组prodID并一起rate并发送。

Answer 2

最好的方法是编写CustomWritables

这是双重价值。 您可以将其更改为文本或字符串

 import java.io.DataInput; import java.io.DataOutput; import java.io.IOException; import org.apache.hadoop.io.Writable; /** * @author Unmesha SreeVeni UB * */ public class TwovalueWritable implements Writable { private double first; private double second; public TwovalueWritable() { set(first, second); } public TwovalueWritable(double first, double second) { set(first, second); } public void set(double first, double second) { this.first = first; this.second = second; } public double getFirst() { return first; } public double getSecond() { return second; } @Override public void write(DataOutput out) throws IOException { out.writeDouble(first); out.writeDouble(second); } @Override public void readFields(DataInput in) throws IOException { first = in.readDouble(); second = in.readDouble(); } /* (non-Javadoc) * @see java.lang.Object#hashCode() */ @Override public int hashCode() { final int prime = 31; int result = 1; long temp; temp = Double.doubleToLongBits(first); result = prime * result + (int) (temp ^ (temp >>> 32)); temp = Double.doubleToLongBits(second); result = prime * result + (int) (temp ^ (temp >>> 32)); return result; } /* (non-Javadoc) * @see java.lang.Object#equals(java.lang.Object) */ @Override public boolean equals(Object obj) { if (this == obj) { return true; } if (obj == null) { return false; } if (!(obj instanceof TwovalueWritable)) { return false; } TwovalueWritable other = (TwovalueWritable) obj; if (Double.doubleToLongBits(first) != Double .doubleToLongBits(other.first)) { return false; } if (Double.doubleToLongBits(second) != Double .doubleToLongBits(other.second)) { return false; } return true; } @Override public String toString() { return first + "," + second; } } 

从mapper你可以发出它

 context.write(key,new TwovalueWritable(prodID,rate)); 

希望这可以帮助。