Hi There,
I have followed all the steps in the installation document and installed the spark job server on CDH and also the clients side extension on Analytic platform. On CDH side I have configured Spark to run on YARN and on KNIME side I have configured the preferences accordinly (attached screenshot) and also changed this line in environment.conf file master = "yarn-client"
When I run an example workflow with spark and its gives me error:"Execute failed: Permission denied: user=root, access=WRITE, inode="/user":hdfs:supergroup:drwxr-xr-x" .
I know this error is because the root does not have permission on /user directory and hdfs is the owner so,
1) How can I connect to spark with a different username apart from "root" from KNIME ? (i donot have kerberos enabled on CDH ) I do not want to change the owner of the user directory.
2) Is there anything that I'm missing to configure. ?
Appreciate immediate help. Thanks !
Regards,
Mohammed Ayub