Share using NFS a volume mounted with gcsfuse in Kubernetes - docker

Question

I've been trying to mount a Google Cloud bucket inside a pod in our (onprem) cluster in order to share that mounted volume using NFS to other pods and PersistentVolumes.
Here there are the configurations:
#!/bin/bash
_start_nfs() {
exportfs -a
rpcbind
rpc.statd
rpc.nfsd
rpc.mountd
GOOGLE_APPLICATION_CREDENTIALS=/accounts/key.json gcsfuse -o allow_other --dir-mode 777 --uid 1500 --gid 1500 ${BUCKET} /exports
}
_nfs_server_mounts() {
IFS=':' read -r -a MNT_SERVER_ARRAY <<< "$NFS_SERVER_DIRS"
for server_mnt in "${MNT_SERVER_ARRAY[#]}"; do
if [[ ! -d $server_mnt ]]; then
mkdir -p $server_mnt
fi
chmod -R 777 $server_mnt
cat >> /etc/exports <<EOF
${server_mnt} *(rw,sync,no_subtree_check,all_squash,anonuid=1500,anongid=1500,fsid=$(( ( RANDOM % 100 ) + 200 )))
EOF
done
cat /etc/exports
}
_sysconfig_nfs() {
cat > /etc/sysconfig/nfs <<EOF
#
#
# To set lockd kernel module parameters please see
# /etc/modprobe.d/lockd.conf
#
# Optional arguments passed to rpc.nfsd. See rpc.nfsd(8)
RPCNFSDARGS=""
# Number of nfs server processes to be started.
# The default is 8.
RPCNFSDCOUNT=${RPCNFSDCOUNT}
#
# Set V4 grace period in seconds
#NFSD_V4_GRACE=90
#
# Set V4 lease period in seconds
#NFSD_V4_LEASE=90
#
# Optional arguments passed to rpc.mountd. See rpc.mountd(8)
RPCMOUNTDOPTS=""
# Port rpc.mountd should listen on.
#MOUNTD_PORT=892
#
# Optional arguments passed to rpc.statd. See rpc.statd(8)
STATDARG=""
# Port rpc.statd should listen on.
#STATD_PORT=662
# Outgoing port statd should used. The default is port
# is random
#STATD_OUTGOING_PORT=2020
# Specify callout program
#STATD_HA_CALLOUT="/usr/local/bin/foo"
#
#
# Optional arguments passed to sm-notify. See sm-notify(8)
SMNOTIFYARGS=""
#
# Optional arguments passed to rpc.idmapd. See rpc.idmapd(8)
RPCIDMAPDARGS=""
#
# Optional arguments passed to rpc.gssd. See rpc.gssd(8)
# Note: The rpc-gssd service will not start unless the
# file /etc/krb5.keytab exists. If an alternate
# keytab is needed, that separate keytab file
# location may be defined in the rpc-gssd.service's
# systemd unit file under the ConditionPathExists
# parameter
RPCGSSDARGS=""
#
# Enable usage of gssproxy. See gssproxy-mech(8).
GSS_USE_PROXY="yes"
#
# Optional arguments passed to blkmapd. See blkmapd(8)
BLKMAPDARGS=""
EOF
}
### main ###
_sysconfig_nfs
_nfs_server_mounts
_start_nfs
rpcinfo -p
showmount -e
tail -f /dev/null
The Dockerfile:
FROM centos:7
RUN yum -y install /usr/bin/ps nfs-utils nfs4-acl-tools curl portmap fuse nfs-utils && yum clean all
RUN mkdir -p /exports
ENV RPCNFSDCOUNT=8 \
NFS_SERVER_DIRS='/usr'
RUN chown nfsnobody:nfsnobody /exports
RUN chmod 777 /exports
ADD setup.sh /usr/local/bin/run_nfs.sh
RUN chmod +x /usr/local/bin/run_nfs.sh
RUN useradd -u 1500 orenes
ADD gcsfuse-0.41.6-1.x86_64.rpm /data/gcsfuse-0.41.6-1.x86_64.rpm
RUN yum install -y /data/gcsfuse-0.41.6-1.x86_64.rpm
# Expose volume
VOLUME ["/exports"]
# expose mountd 20048/tcp and nfsd 2049/tcp and rpcbind 111/tcp
EXPOSE 2049/tcp 20048/tcp 111/tcp 111/udp
ENTRYPOINT ["/usr/local/bin/run_nfs.sh"]
Let's assume that there's a PersistentVolume mounting a Service pointing to the NFS server I described before.
So far I got access denied, unable to receive, etc...
Should I move to a SMB/CIFS sharing? Is there any solution on this? Thanks in advance.

Answer 1

It seems that FUSE and NFS don't get well along. I ended up using SMB and it works flawlessly but you must use the Service ClusterIP instead of Kubernetes's DNS.
The comment from #AviD about using a CSI is so helpful if you need to make it quick.
Again thanks to all.