Oozie 와의 연동

1. Ambari로 Oozie 설치 후 아래와 같이 설정

- custom oozie-site

oozie.action.jobinfo.enable = true

oozie.action.launcher.yarn.timeline-service.enabled = false

oozie.service.ProxyUserService.proxyuser.root.groups = *

oozie.service.ProxyUserService.proxyuser.root.hosts = *

- Advanced oozie-site

oozie.service.AuthorizationService.security.enabled=false

2. oozie관련 xml을 작성하여 지정된 위치에 업로드

(hdfs://node01:8020/user/john/oozie-sample.xml)

<workflow-app xmlns="uri:oozie:workflow:0.5" xmlns:sla="uri:oozie:sla:0.2" name="tenant_test">
    <global>
        <job-tracker>${resourceManager}</job-tracker>
        <name-node>${nameNode}</name-node>
    </global>
    <start to="hdfs8"/>
    <action name="hdfs8">
        <ssh xmlns="uri:oozie:ssh-action:0.1">
            <host>root@node01</host>
            <command>/home/root/apps/bin/run-oozie-sample-mr.sh</command>
            <args>${wf:user()}</args>
            <args>/user/${wf:user()}/work</args>
        </ssh>
        <ok to="end"/>
        <error to="fail"/>
    </action>
    <kill name="fail">
        <message>FAIL</message>
    </kill>
    <end name="end"/>
</workflow-app>

3. xml에서 사용할 property 설정

(/home/root/apps/config/example.properties)


# Set the Name Node URI e.g. hdfs://sandbox.hortonworks.com:8020
name.node=hdfs://node01:8020
# Set the Resource Manager URI e.g. sandbox.hortonworks.com:8050
resource.manager=node02:8050
nameNode=${name.node}
jobTracker=${resource.manager}
oozie.wf.application.path=hdfs://node01:8020/user/john/oozie-sample.xml
oozie.use.system.libpath=true

4. oozie 실행

oozie job -oozie http://node01:11000/oozie -config example.properties -doas john -run

5. 참고

user 변경

Oozie client 로 job submit 을 할 때는 user.name 속성이 적용되지 않음(os 계정)
user.name oozie command 옵션 중 -doas <userid> 를 사용

group 변경

Job submit 시 properties 파일에 oozie.job.acl 설정 추가
예: oozie.job.acl=<groupId>