从sql server导入数据的Sqoop作业忽略模式
如何将模式名称传递给sqoop job将数据从SQL Server导入到hdfs ？
sqoop job --create job_name -- import --connect "jdbc:sqlserver://server:port;database=datatabase_name;username=user;password=password" --table source_table --as-avrodatafile --target-dir data/target_folder -- --schema schema_name
当我使用执行作业时
sqoop job -exec job_name
生成的查询缺少模式名称。
它失败并显示以下错误消息：
15/08/28 10:53:09 INFO manager.SqlManager: Executing SQL statement: **SELECT t.* FROM [source_table] AS t WHERE 1=0** 15/08/28 10:53:09 ERROR manager.SqlManager: Error executing statement: com.microsoft.sqlserver.jdbc.SQLServerException: Invalid object name 'source_table'. com.microsoft.sqlserver.jdbc.SQLServerException: Invalid object name 'source_table'. at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDatabaseError(SQLServerException.java:216) at com.microsoft.sqlserver.jdbc.SQLServerStatement.getNextResult(SQLServerStatement.java:1515) at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.doExecutePreparedStatement(SQLServerPreparedStatement.java:404) at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement$PrepStmtExecCmd.doExecute(SQLServerPreparedStatement.java:350) at com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:5696) at com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:1715) at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeCommand(SQLServerStatement.java:180) at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeStatement(SQLServerStatement.java:155) at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.executeQuery(SQLServerPreparedStatement.java:285) at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:750) at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:759) at org.apache.sqoop.manager.SqlManager.getColumnInfoForRawQuery(SqlManager.java:269) at org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:240) at org.apache.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:226) at org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:295) at org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1773) at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1578) at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:96) at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:478) at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:601) at org.apache.sqoop.tool.JobTool.execJob(JobTool.java:228) at org.apache.sqoop.tool.JobTool.run(JobTool.java:283) at org.apache.sqoop.Sqoop.run(Sqoop.java:143) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227) at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
有什么建议么？
我遇到了同样的问题，我使用了以下命令，
sqoop job --exec job_name -- -- --schema schema_name
尝试这个 ：
sqoop job --create job_name -- import --connect "jdbc:sqlserver://server:port;database=datatabase_name;username=user;password=password" --table source_table --as-avrodatafile --target-dir data/target_folder -- --schema schema_name --verbose --columns .....list of columns here(comma separated)
在Pradeep的答案上小猪支持，这是我在作业模式中使用schema命令创建作业的方式：
sqoop job --create job_name -- import --connect "jdbc:sqlserver://server:port;database=datatabase_name;username=user;password=password" --table source_table --as-avrodatafile --target-dir data/target_folder -- -- --schema schema_name
请注意，最后有三组–最后一组。 -- -- --schema schema_name
我使用它来自动创建执行增量更新的作业。 为了利用sqoop自动跟踪增量更新的能力，我需要使用一份工作。