run Airflow webserver on EC2 Centos using Systemd: Permission denied - amazon-ec2

Question

I'm using Airflow in EC2 CentOS with systemd to manage daemons for starting airflow processes (ie: webserver, worker and scheduler).
When I run sudo systemctl start airflow-webserver
: ec2-user : TTY=pts/0 ; PWD=/ ; USER=root ; COMMAND=/bin/systemctl enable airflow-webserver
Oct 30 01:27:37 ip-171-32-6-92.us-east-2.compute.internal sudo[11680]: ec2-user : TTY=pts/0 ; PWD=/ ; USER=root ; COMMAND=/bin/systemctl start airflow-webserver
Oct 30 01:27:37 ip-171-32-6-92.us-east-2.compute.internal systemd[11684]: Failed at step EXEC spawning /home/ec2-user/.local/bin/airflow: Permission denied
Oct 30 01:27:37 ip-171-32-6-92.us-east-2.compute.internal systemd[1]: airflow-webserver.service: main process exited, code=exited, status=203/EXEC
Oct 30 01:27:37 ip-171-32-6-92.us-east-2.compute.internal systemd[1]: Unit airflow-webserver.service entered failed state.
Oct 30 01:27:37 ip-171-32-6-92.us-east-2.compute.internal systemd[1]: airflow-webserver.service failed
airflow-webserver.service
[Unit]
Description=Airflow webserver daemon
After=network.target postgresql.service mysql.service redis.service rabbitmq-server.service
Wants=postgresql.service mysql.service redis.service rabbitmq-server.service
[Service]
EnvironmentFile=/etc/sysconfig/airflow
User=airflow
Group=airflow
Type=simple
ExecStart=/home/ec2-user/.local/bin/airflow webserver --pid /run/airflow/webserver.pid
RestartSec=5s
PrivateTmp=true
[Install]
WantedBy=multi-user.target
airflow group and user are present:
$ less /etc/group
# airflow:x:1001:ec2-user
$less etc/psswd
# ec2-user:x:1000:1000:EC2 Default User:/home/ec2-user:/bin/bash
# tss:x:59:59:Account used by the trousers package to sandbox the tcsd # daemon:/dev/null:/sbin/nologin
# airflow:x:1001:1001::/home/airflow:/bin/bash
THIS does not help.
UPDATE
after creating an airflow user and installing airflow under usr/local/bin then changing the ExecStart to usr/local/bin/airflow webserver --pid /run/airflow/webserver.pid I was able to get passed the initial permission denied. However now I get this error:
Oct 31 18:44:12 ip-172-31-6-92.us-east-2.compute.internal airflow[31430]: PermissionError: [Errno 13] Permission denied: '${AIRFLOW_HOME}'
Oct 31 18:44:12 ip-172-31-6-92.us-east-2.compute.internal airflow[31430]: During handling of the above exception, another exception occurred:
Oct 31 18:44:12 ip-172-31-6-92.us-east-2.compute.internal airflow[31430]: Traceback (most recent call last):
Oct 31 18:44:12 ip-172-31-6-92.us-east-2.compute.internal airflow[31430]: File "/usr/local/bin/airflow", line 25, in <module>
Oct 31 18:44:12 ip-172-31-6-92.us-east-2.compute.internal airflow[31430]: from airflow.configuration import conf
Oct 31 18:44:12 ip-172-31-6-92.us-east-2.compute.internal airflow[31430]: File "/usr/local/lib/python3.7/site-packages/airflow/__init__.py", line 31, in <module>
Oct 31 18:44:12 ip-172-31-6-92.us-east-2.compute.internal airflow[31430]: from airflow.utils.log.logging_mixin import LoggingMixin
Oct 31 18:44:12 ip-172-31-6-92.us-east-2.compute.internal airflow[31430]: File "/usr/local/lib/python3.7/site-packages/airflow/utils/__init__.py", line 24, in <module>
Oct 31 18:44:12 ip-172-31-6-92.us-east-2.compute.internal airflow[31430]: from .decorators import apply_defaults as _apply_defaults
Oct 31 18:44:12 ip-172-31-6-92.us-east-2.compute.internal airflow[31430]: File "/usr/local/lib/python3.7/site-packages/airflow/utils/decorators.py", line 34, in <module>
Oct 31 18:44:12 ip-172-31-6-92.us-east-2.compute.internal airflow[31430]: from airflow import settings
Oct 31 18:44:12 ip-172-31-6-92.us-east-2.compute.internal airflow[31430]: File "/usr/local/lib/python3.7/site-packages/airflow/settings.py", line 36, in <module>
Oct 31 18:44:12 ip-172-31-6-92.us-east-2.compute.internal airflow[31430]: from airflow.configuration import conf, AIRFLOW_HOME, WEBSERVER_CONFIG # NOQA F401
Oct 31 18:44:12 ip-172-31-6-92.us-east-2.compute.internal airflow[31430]: File "/usr/local/lib/python3.7/site-packages/airflow/configuration.py", line 523, in <module>
Oct 31 18:44:12 ip-172-31-6-92.us-east-2.compute.internal airflow[31430]: mkdir_p(AIRFLOW_HOME)
Oct 31 18:44:12 ip-172-31-6-92.us-east-2.compute.internal airflow[31430]: File "/usr/local/lib/python3.7/site-packages/airflow/configuration.py", line 505, in mkdir_p
Oct 31 18:44:12 ip-172-31-6-92.us-east-2.compute.internal airflow[31430]: 'Error creating {}: {}'.format(path, exc.strerror))
Oct 31 18:44:12 ip-172-31-6-92.us-east-2.compute.internal airflow[31430]: airflow.exceptions.AirflowConfigException: Error creating ${AIRFLOW_HOME}: Permission denied

Answer 1

This is because you are trying to run as airflow user but that path to airflow binary is in /home/ec2-user/.local/bin/airflow which is a HOME directory for ec2-user.
Install airflow in a directory where your airflow users have permission to.