hadoop中job.setOutputFormatClass(PartitionByCoun...
生活随笔
收集整理的這篇文章主要介紹了
hadoop中job.setOutputFormatClass(PartitionByCoun...
小編覺得挺不錯的,現(xiàn)在分享給大家,幫大家做個參考.
2019獨角獸企業(yè)重金招聘Python工程師標準>>>
import java.io.IOException; import org.apache.hadoop.conf.Configuration; import org.apache.hadoop.conf.Configured; import org.apache.hadoop.fs.Path; import org.apache.hadoop.io.*; import org.apache.hadoop.mapred.JobClient; import org.apache.hadoop.mapred.OutputCollector; import org.apache.hadoop.mapred.Reporter; import org.apache.hadoop.mapred.lib.MultipleTextOutputFormat; import org.apache.hadoop.mapreduce.*; import org.apache.hadoop.mapreduce.lib.input.FileInputFormat; import org.apache.hadoop.mapreduce.lib.input.TextInputFormat; import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat; import org.apache.hadoop.mapreduce.lib.output.TextOutputFormat; import org.apache.hadoop.util.Tool; import org.apache.hadoop.util.ToolRunner;public class MultiFile extends Configured {public static class MapClass extends Mapper<LongWritable, Text, NullWritable, Text> {public void map(LongWritable key, Text value,OutputCollector<NullWritable, Text> output,Reporter reporter) throws IOException {output.collect(NullWritable.get(), value);}}public static class PartitionByCountryMTOFextends MultipleTextOutputFormat<NullWritable,Text>{protected String generateFileNameForKeyValue(NullWritable key,Text value,String filename){String[] arr = value.toString().split(",", -1);String country = arr[4].substring(1,3);return country + "/" + filename;}}public static void main(String[] args) throws Exception { Configuration conf = new Configuration();Job job = new Job(conf, "MultiFile");job.setJarByClass(MultiFile.class);Path in = new Path(args[0]);Path out = new Path(args[1]);FileInputFormat.setInputPaths(job, in);//輸入路徑FileOutputFormat.setOutputPath(job, out);//輸出路徑job.setJobName("MultiFile");//job名稱job.setMapperClass(MapClass.class);job.setInputFormatClass(TextInputFormat.class);//輸入格式job.setOutputFormatClass(PartitionByCountryMTOF.class);//輸出格式設置job.setMapOutputValueClass(LongWritable.class);//job.setOutputKeyClass(NullWritable.class);輸出key的類型job.setOutputValueClass(Text.class);//輸出value的類型job.setNumReduceTasks(0);//設置reduce個數(shù)System.exit(job.waitForCompletion(true)?0:1);} }job.setOutputFormatClass(PartitionByCountryMTOF.class);
顯示The method setOutputFormatClass(Class<? extends OutputFormat>) in the type Job is?
?not applicable for the arguments (Class<MultiFile.PartitionByCountryMTOF>)
請問怎么解決??
轉載于:https://my.oschina.net/u/242736/blog/83042
總結
以上是生活随笔為你收集整理的hadoop中job.setOutputFormatClass(PartitionByCoun...的全部內容,希望文章能夠幫你解決所遇到的問題。
- 上一篇: 王者荣耀攻击速度上限是多少(如何玩好《王
- 下一篇: 皇室战争三枪卡组2018是什么(1717