I am using linux os (4.9.126) built with yocto (sumo). It consists of glibc v2.27, util-linux v2.32.1.
The pthread library is located at:
ldconfig -p | grep pthread.so
libpthread.so.0 (libc6,x86-64, OS ABI: Linux 3.2.0) => /lib/libpthread.so.0
I want to set the priority SCHD_RR inside docker containerized C++ application.
..
..
...
int set_max_thread_priority()
{
int policy;
struct sched_param param {
};
pthread_getschedparam(pthread_self(), &policy, ¶m);
policy = SCHED_RR;
param.sched_priority = sched_get_priority_max(policy);
return pthread_setschedparam(pthread_self(), policy, ¶m);
}
...
I want to achieve
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
2083 root rt 0 709812 8100 6456 S 44.4 0.2 0:32.24 my-application
But it shows,
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
2083 root 20 0 709812 8100 6456 S 44.4 0.2 0:32.24 my-application
When I run docker containerized my-application on Ubuntu/Debian, then it works fine.
It does not work with yocto os. When I try to change the thread priority on yocto os (using chrt command) it shows following error.
sudo chrt -r -p 99 2083
chrt: failed to set pid 18465's policy: Operation not permitted
I am not sure if I am missing any package inside OS.
Why don't you use cgroups and create rules for your application's priority.
Related
I tried to use dpdk-pktgen 3.7.2 with dpdk 18.11, but it only reached about 35 Mpps traffic with 64B packet size.Following is my lua script:
package.path = package.path ..";?.lua;test/?.lua;app/?.lua;../?.lua"
require "Pktgen";
local time = 30;
local pcnt_rate = 100;
sendport = 0;
recvport = 1;
pkt_size = 64;
burst_cnt = 128
local dstip = "192.168.100.100";
local srcip = "192.168.0.0";
function main()
pktgen.stop(sendport);
sleep(2);
pktgen.set(sendport, "size", burst_cnt);
pktgen.set(sendport, "burst", 64);
pktgen.set(sendport, "rate", pcnt_rate);
pktgen.set_ipaddr(sendport, "dst", dstip);
pktgen.set_ipaddr(sendport, "src", srcip);
pktgen.set_proto(sendport..","..recvport, "udp");
pktgen.start(sendport)
sleep(time)
pktgen.stop(sendport)
end
printf("\n**** Traffic Profile Rate for %d byte packets ***\n", pkt_size);
main();
printf("\n*** Traffic Profile Done (Total Time %d) ***\n", time);
l ran the script with the following command.
sudo pktgen -l 0-7 -n 4 -- -N -T -P -m "[1-7].0" -f script.lua
My NIC is Mellanox ConnectX-5 100GbE with traffic limit of 200Mpps and 100Gbps. Is there any problem in my script that restricts performance of pktgen? Thank you for your suggestions.
As mentioned in comments this is more of configuration issue of platform or not choosing right platform. I am able to generate 120Mpps with 64B on 100Gbps (CVL NIC).
[EDIT-2] Finally got hands on Mellanox connectx6 DX cards (2 *100Gbps). With DPDK 21.11 and PKTGEN 21.11, it is possible to generate over 100MPPs
MLX PMD args used: mprq_en=1,rxqs_min_mprq=1,mprq_log_stride_num=9,txq_inline_mpw=128,rxq_pkt_pad_en=1
Platform Details:
DPDK:21.08
PKTGEN:pktgen-dpdk-pktgen-21.03.1
NIC:Ethernet Controller E810-C for QSFP
CPU: Intel(R) Xeon(R) Gold 6152 CPU # 2.10GHz
PKTGEN CMD:pktgen --legacy-mem -a 0000:86:00.0 -l 22-43 -- -P -m "[25-29:30-34].0" -N
[EDIT-1] #SoliRaven as mentioned in comments and answer one can generate up to 120 to 125 Mpps with Intel E810 & with Mellanox Connect6 DX 120MPPs. Hence this looks more like configuration, platform or firmware issues. Hence this is not DPDK or DPDK-PKTGEN is not issue.
our dask scheduler process seems to balloon in memory as time goes on and executions continue. Currently we see it using 5GB of mem, which seems high since all the data is supposedly living on the worker nodes:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
31172 atoz 20 0 5486944 5.071g 7100 S 23.8 65.0 92:38.64 dask-scheduler
when starting up the scheduler we would be below 1GB of memory use. Restarting the network doing a client.restart() doesn't seem to help, only a kill of the scheduler process itself and restart will free up the memory.
What is the expected usage of memory per single task executed?
Is the scheduler really only maintaining pointers to which worker contains the future's result?
----edit----
I think my main concern here is why a client.restart() doesn't seem to release the memory being used by the scheduler process. I'm obviously not expecting it to release all memory, but to get back to a base level. We are using client.map to execute our function across a list of different inputs. After executing, doing a client restart over and over and taking snapshots of our scheduler memory we see the following growth:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
27955 atoz 20 0 670556 507212 13536 R 43.7 6.2 1:23.61 dask-scheduler
27955 atoz 20 0 827308 663772 13536 S 1.7 8.1 16:25.85 dask-scheduler
27955 atoz 20 0 859652 696408 13536 S 4.0 8.5 19:18.04 dask-scheduler
27955 atoz 20 0 1087160 923912 13536 R 62.3 11.3 20:03.15 dask-scheduler
27955 atoz 20 0 1038904 875788 13536 S 3.7 10.7 23:57.07 dask-scheduler
27955 atoz 20 0 1441060 1.163g 12976 S 4.3 14.9 35:54.45 dask-scheduler
27955 atoz 20 0 1646204 1.358g 12976 S 4.3 17.4 37:05.86 dask-scheduler
27955 atoz 20 0 1597652 1.312g 12976 S 4.7 16.8 37:40.13 dask-scheduler
I guess I was just surprised that after doing a client.restart() we don't see the memory usage go back to some baseline.
----further edits----
Some more info about what we're running, since the suggestion was if we were passing in large data structures, to send them directly to the workers.
we send a dictionary as an input for each task, when json dumping the dict, most are under 1000 characters.
---- even further edits: Reproduced issue ----
We reproduced this issue again today. I killed off the scheduler and restarted it, we had about 5.4 GB of free memory, we then ran the function that I'll paste below across 69614 dictionary objects that really hold some file based information (all of our workers are mapped to the same NFS datastore and we are using Dask as a distributed file analysis system.
Here is the function (note: squarewheels4 is a homegrown lazy file extraction and analysis package, it uses Acora and libarchive as its base for getting files out of a compressed archive and indexing the file.)
def get_mrc_failures(file_dict):
from squarewheels4.platforms.ucs.b_series import ChassisTechSupport
from squarewheels4.files.ucs.managed.chassis import CIMCTechSupportFile
import re
dimm_info_re = re.compile(r"(?P<slot>[^\|]+)\|(?P<size>\d+)\|.*\|(?P<pid>\S+)")
return_dict = file_dict
return_dict["return_code"] = "NOT_FILLED_OUT"
filename = "{file_path}{file_sha1}/{file_name}".format(**file_dict)
try:
sw = ChassisTechSupport(filename)
except Exception as e:
return_dict["return_code"] = "SW_LOAD_ERROR"
return_dict["error_msg"] = str(e)
return return_dict
server_dict = {}
cimcs = sw.getlist("CIMC*.tar.gz")
if not cimcs:
return_dict["return_code"] = "NO_CIMCS"
return_dict["keys_list"] = str(sw.getlist("*"))
return return_dict
for cimc in cimcs:
if not isinstance(cimc, CIMCTechSupportFile): continue
cimc_id = cimc.number
server_dict[cimc_id] = {}
# Get MRC file
try:
mrc = cimc["*MrcOut.txt"]
except KeyError:
server_dict[cimc_id]["response_code"] = "NO_MRC"
continue
# see if our end of file marker is there, should look like:
# --- END OF FILE (Done!
whole_mrc = mrc.read().splitlines()
last_10 = whole_mrc[-10:]
eof_line = [l for l in last_10 if b"END OF FILE" in l]
server_dict[cimc_id]["response_code"] = "EOF_FOUND" if eof_line else "EOF_MISSING"
if eof_line:
continue
# get DIMM types
hit_inventory_line = False
dimm_info = []
dimm_error_lines = []
equals_count = 0
for line in whole_mrc:
# regex each line... sigh
if b"DIMM Inventory" in line:
hit_inventory_line = True
if not hit_inventory_line:
continue
if hit_inventory_line and b"=========" in line:
equals_count += 1
if equals_count > 2:
break
continue
if equals_count < 2:
continue
# we're in the dimm section and not out of it yet
line = str(line)
reg = dimm_info_re.match(line)
if not reg:
#bad :/
dimm_error_lines.append(line)
continue
dimm_info.append(reg.groupdict())
server_dict[cimc_id]["dimm_info"] = dimm_info
server_dict[cimc_id]["dimm_error_lines"] = dimm_error_lines
return_dict["return_code"] = "COMPLETED"
return_dict["server_dict"] = server_dict
return return_dict
```
the futures are generated like:
futures = client.map(function_name, file_list)
After in this state my goal was to try and recover and have dask release the memory that it had allocated, here were my efforts:
before cancelling futures:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
21914 atoz 20 0 6257840 4.883g 2324 S 0.0 62.6 121:21.93 dask-scheduler
atoz#atoz-sched:~$ free -h
total used free shared buff/cache available
Mem: 7.8G 7.1G 248M 9.9M 415M 383M
Swap: 8.0G 4.3G 3.7G
while cancelling futures:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
21914 atoz 20 0 6258864 5.261g 5144 R 60.0 67.5 122:16.38 dask-scheduler
atoz#atoz-sched:~$ free -h
total used free shared buff/cache available
Mem: 7.8G 7.5G 176M 9.4M 126M 83M
Swap: 8.0G 4.1G 3.9G
after cancelling futures:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
21914 atoz 20 0 6243760 5.217g 4920 S 0.0 66.9 123:13.80 dask-scheduler
atoz#atoz-sched:~$ free -h
total used free shared buff/cache available
Mem: 7.8G 7.5G 186M 9.4M 132M 96M
Swap: 8.0G 4.1G 3.9G
after doing a client.restart()
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
21914 atoz 20 0 6177424 5.228g 4912 S 2.7 67.1 123:20.04 dask-scheduler
atoz#atoz-sched:~$ free -h
total used free shared buff/cache available
Mem: 7.8G 7.5G 196M 9.4M 136M 107M
Swap: 8.0G 4.0G 4.0G
Regardless of what I ran through the distributed system, my expectation was that after cancelling the futures it would be back to at least close to normal... and after doing a client.restart() we would definitely be near our normal baseline. Am I wrong here?
--- second repro ----
Reproduced the behavior (although not total memory exhaustion) using these steps:
Here's my worker function
def get_fault_list_v2(file_dict):
import libarchive
return_dict = file_dict
filename = "{file_path}{file_sha1}/{file_name}".format(**file_dict)
with libarchive.file_reader(filename) as arc:
for e in arc:
pn = e.pathname
return return_dict
I ran that across 68617 iterations / files
before running we saw this much memory being utilized:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
12256 atoz 20 0 1345848 1.107g 7972 S 1.7 14.2 47:15.24 dask-scheduler
atoz#atoz-sched:~$ free -h
total used free shared buff/cache available
Mem: 7.8G 3.1G 162M 22M 4.5G 4.3G
Swap: 8.0G 3.8G 4.2G
After running we saw this much:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
12256 atoz 20 0 2461004 2.133g 8024 S 1.3 27.4 66:41.46 dask-scheduler
After doing a client.restart we saw:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
12256 atoz 20 0 2462756 2.134g 8144 S 6.6 27.4 66:42.61 dask-scheduler
Generally a task should take up less than a kilobyte on the scheduler. There are a few things you can trip up on that result in storing significantly more, the most common of which is including data within the task graph, which is shown below.
Data included directly in a task graph is stored on the scheduler. This commonly occurs when using large data directly in calls like submit:
Bad
x = np.random.random(1000000) # some large array
future = client.submit(np.add, 1, x) # x gets sent along with the task
Good
x = np.random.random(1000000) # some large array
x = client.scatter(x) # scatter data explicitly to worker, get future back
future = client.submit(np.add, 1, x) # only send along the future
This same principle exists using other APIs as well. For more information, I recommend providing an mcve. It's quite hard to help otherwise.
I have some resources used by a specific user that I had to delete because it was taking a lot of resources from the server. When I listed the processes in the server the deleted user now shows as “1001” instead of the name it used to show before I deleted it.
%Cpu(s): 19.8 us, 29.5 sy, 0.0 ni, 50.7 id, 0.0 wa, 0.0 hi, 0.1 si, 0.0 st
KiB Mem : 3882456 total, 183568 free, 2003808 used, 1695080 buff/cache
KiB Swap: 1679356 total, 1155300 free, 524056 used. 1463480 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
9192 1001 20 0 2068436 74700 10284 S 0.3 1.9 3:02.86 node
By using systemd status I found the process and the docker container ID that the user is executing is in.
├─docker
│ ├─42b40e73687acb7fcd9a0e43372ced7588b5568c942f740d06510ab0e85b1462
│ │ ├─17156 /bin/sh -e /usr/local/sbin/start.sh
└─11148 node --debug --nolazy dist-release/server
So I went into the container and I look to the start.sh file but it’s just an executable file, there’s no indication inside of the file that the user is getting called inside of the executable file.
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
apiassets_1
42b40e73687a local.io/api-statements:development "start.sh" 21 hours ago Up 18 hours 0.0.0.0:32785->3000/tcp, 0.0.0.0:5966->5858/tcp
What I want to do is stopping this user to use this resources, so I was just curious how can I either find how this user is calling this script to stop it or how can I stop it.
I submit my code to a spark stand alone cluster. Submit command is like below:
nohup ./bin/spark-submit \
--master spark://ES01:7077 \
--executor-memory 4G \
--num-executors 1 \
--total-executor-cores 1 \
--conf "spark.storage.memoryFraction=0.2" \
./myCode.py 1>a.log 2>b.log &
I specify the executor use 4G memory in above command. But use the top command to monitor the executor process, I notice the memory usage keeps growing. Now the top Command output is below:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
12578 root 20 0 20.223g 5.790g 23856 S 61.5 37.3 20:49.36 java
My total memory is 16G so 37.3% is already bigger than the 4GB I specified. And it is still growing.
Use the ps command , you can know it is the executor process.
[root#ES01 ~]# ps -awx | grep spark | grep java
10409 ? Sl 1:43 java -cp /opt/spark-1.6.0-bin-hadoop2.6/conf/:/opt/spark-1.6.0-bin-hadoop2.6/lib/spark-assembly-1.6.0-hadoop2.6.0.jar:/opt/spark-1.6.0-bin-hadoop2.6/lib/datanucleus-api-jdo-3.2.6.jar:/opt/spark-1.6.0-bin-hadoop2.6/lib/datanucleus-rdbms-3.2.9.jar:/opt/spark-1.6.0-bin-hadoop2.6/lib/datanucleus-core-3.2.10.jar:/opt/hadoop-2.6.2/etc/hadoop/ -Xms4G -Xmx4G -XX:MaxPermSize=256m org.apache.spark.deploy.master.Master --ip ES01 --port 7077 --webui-port 8080
10603 ? Sl 6:16 java -cp /opt/spark-1.6.0-bin-hadoop2.6/conf/:/opt/spark-1.6.0-bin-hadoop2.6/lib/spark-assembly-1.6.0-hadoop2.6.0.jar:/opt/spark-1.6.0-bin-hadoop2.6/lib/datanucleus-api-jdo-3.2.6.jar:/opt/spark-1.6.0-bin-hadoop2.6/lib/datanucleus-rdbms-3.2.9.jar:/opt/spark-1.6.0-bin-hadoop2.6/lib/datanucleus-core-3.2.10.jar:/opt/hadoop-2.6.2/etc/hadoop/ -Xms4G -Xmx4G -XX:MaxPermSize=256m org.apache.spark.deploy.worker.Worker --webui-port 8081 spark://ES01:7077
12420 ? Sl 10:16 java -cp /opt/spark-1.6.0-bin-hadoop2.6/conf/:/opt/spark-1.6.0-bin-hadoop2.6/lib/spark-assembly-1.6.0-hadoop2.6.0.jar:/opt/spark-1.6.0-bin-hadoop2.6/lib/datanucleus-api-jdo-3.2.6.jar:/opt/spark-1.6.0-bin-hadoop2.6/lib/datanucleus-rdbms-3.2.9.jar:/opt/spark-1.6.0-bin-hadoop2.6/lib/datanucleus-core-3.2.10.jar:/opt/hadoop-2.6.2/etc/hadoop/ -Xms1g -Xmx1g -XX:MaxPermSize=256m org.apache.spark.deploy.SparkSubmit --master spark://ES01:7077 --conf spark.storage.memoryFraction=0.2 --executor-memory 4G --num-executors 1 --total-executor-cores 1 /opt/flowSpark/sparkStream/ForAsk01.py
12578 ? Sl 21:03 java -cp /opt/spark-1.6.0-bin-hadoop2.6/conf/:/opt/spark-1.6.0-bin-hadoop2.6/lib/spark-assembly-1.6.0-hadoop2.6.0.jar:/opt/spark-1.6.0-bin-hadoop2.6/lib/datanucleus-api-jdo-3.2.6.jar:/opt/spark-1.6.0-bin-hadoop2.6/lib/datanucleus-rdbms-3.2.9.jar:/opt/spark-1.6.0-bin-hadoop2.6/lib/datanucleus-core-3.2.10.jar:/opt/hadoop-2.6.2/etc/hadoop/ -Xms4096M -Xmx4096M -Dspark.driver.port=52931 -XX:MaxPermSize=256m org.apache.spark.executor.CoarseGrainedExecutorBackend --driver-url spark://CoarseGrainedScheduler#10.79.148.184:52931 --executor-id 0 --hostname 10.79.148.184 --cores 1 --app-id app-20160511080701-0013 --worker-url spark://Worker#10.79.148.184:52660
Below are the code. It is very simple so I do not think there is memory leak
if __name__ == "__main__":
dataDirectory = '/stream/raw'
sc = SparkContext(appName="Netflow")
ssc = StreamingContext(sc, 20)
# Read CSV File
lines = ssc.textFileStream(dataDirectory)
lines.foreachRDD(process)
ssc.start()
ssc.awaitTermination()
The code for process function is below. Please note that I am using HiveContext not SqlContext here. Because SqlContext do not support window function
def getSqlContextInstance(sparkContext):
if ('sqlContextSingletonInstance' not in globals()):
globals()['sqlContextSingletonInstance'] = HiveContext(sparkContext)
return globals()['sqlContextSingletonInstance']
def process(time, rdd):
if rdd.isEmpty():
return sc.emptyRDD()
sqlContext = getSqlContextInstance(rdd.context)
# Convert CSV File to Dataframe
parts = rdd.map(lambda l: l.split(","))
rowRdd = parts.map(lambda p: Row(router=p[0], interface=int(p[1]), flow_direction=p[9], bits=int(p[11])))
dataframe = sqlContext.createDataFrame(rowRdd)
# Get the top 2 interface of each router
dataframe = dataframe.groupBy(['router','interface']).agg(func.sum('bits').alias('bits'))
windowSpec = Window.partitionBy(dataframe['router']).orderBy(dataframe['bits'].desc())
rank = func.dense_rank().over(windowSpec)
ret = dataframe.select(dataframe['router'],dataframe['interface'],dataframe['bits'], rank.alias('rank')).filter("rank<=2")
ret.show()
dataframe.show()
Actually I found below code will cause the problem:
# Get the top 2 interface of each router
dataframe = dataframe.groupBy(['router','interface']).agg(func.sum('bits').alias('bits'))
windowSpec = Window.partitionBy(dataframe['router']).orderBy(dataframe['bits'].desc())
rank = func.dense_rank().over(windowSpec)
ret = dataframe.select(dataframe['router'],dataframe['interface'],dataframe['bits'], rank.alias('rank')).filter("rank<=2")
ret.show()
Because If I remove these 5 line. The code can run all night without showing memory increase. But adding them will cause the memory usage of executor grow to a very high number.
Basically the above code is just some window + grouby in SparkSQL. So is this a bug?
Disclaimer: this answer isn't based on debugging, but more on observations and the documentation Apache Spark provides
I don't believe that this is a bug to begin with!
Looking at your configurations, we can see that you are focusing mostly on the executor tuning, which isn't wrong, but you are forgetting the driver part of the equation.
Looking at the spark cluster overview from Apache Spark documentaion
As you can see, each worker has an executor, however, in your case, the worker node is the same as the driver node! Which frankly is the case when you run locally or on a standalone cluster in a single node.
Further, the driver takes 1G of memory by default unless tuned using spark.driver.memory flag. Furthermore, you should not forget about the heap usage from the JVM itself, and the Web UI that's been taken care of by the driver too AFAIK!
When you delete the lines of code you mentioned, your code is left without actions as map function is just a transformation, hence, there will be no execution, and therefore, you don't see memory increase at all!
Same applies on groupBy as it is just a transformation that will not be executed unless an action is being called which in your case is agg and show further down the stream!
That said, try to minimize your driver memory and the overall number of cores in spark which is defined by spark.cores.max if you want to control the number of cores on this process, then cascade down to the executors. Moreover, I would add spark.python.profile.dump to your list of configuration so you can see a profile for your spark job execution, which can help you more with understanding the case, and to tune your cluster more to your needs.
As I can see in your 5 lines, maybe the groupBy is the issue , would you try with reduceBy, and see how it performs.
See here and here.
I am attempting to run Celery as a Windows Service using Supervisord. I followed the configuration laid out on the Celery site and [here][1]. I have set up a virtual environment to run supervisord through cygwin.I have highlighted the lines I think are most important (with **). It appears supervisord and rabbitMQ are working. The problem is with Celery.
I setup the service with the commands:
$ cygrunsrv --install supervisord --path /usr/bin/python --args "/usr/bin/supervisord -n -c /usr/etc/supervisord.conf"
$ supervisord
UPDATED: I now have the following in my supervisord.log file:
2014-08-07 12:46:40,676 INFO exited: celery (exit status 1; not expected)
2014-08-07 12:47:07,187 INFO Increased RLIMIT_NOFILE limit to 1024
2014-08-07 12:47:07,238 INFO RPC interface 'supervisor' initialized
2014-08-07 12:47:07,251 INFO daemonizing the supervisord process
2014-08-07 12:47:07,253 INFO supervisord started with pid 7508
2014-08-07 12:47:08,272 INFO spawned: 'celery' with pid 8056
**2014-08-07 12:47:08,833 INFO success: celery entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)**
The config file is:
[inet_http_server] ; inet (TCP) server disabled by default
port=127.0.0.1:8072 ; (ip_address:port specifier, *:port for all iface)
username = user
password = 123
[supervisord]
logfile= /home/HBA/venv/logFiles/supervisord.log ; (main log file;default $CWD/supervisord.log)
logfile_maxbytes=50MB ; (max main logfile bytes b4 rotation;default 50MB)
logfile_backups=10 ; (num of main logfile rotation backups;default 10)
loglevel=info ; (log level;default info; others: debug,warn,trace)
pidfile=/tmp/supervisord.pid ; (supervisord pidfile;default supervisord.pid)
nodaemon=false ; (start in foreground if true;default false)
minfds=1024 ; (min. avail startup file descriptors;default 1024)
minprocs=200 ; (min. avail process descriptors;default 200)
;user=HBA ; (default is current user, required if root)
childlogdir=/tmp ; ('AUTO' child log dir, default $TEMP)
[rpcinterface:supervisor]
supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface
[supervisorctl]
serverurl=http://127.0.0.1:8072 ; use an http:// url to specify an inet socket
[program:celery]
command= celery worker -A runLogProject --loglevel=INFO ; the program (relative uses PATH, can take args)
directory= /home/HBA/venv/runLogProject
environment=PATH="/home/HBA/venv/;/home/HBA/venv/Scripts/"
numprocs=1
stdout_logfile= /home/HBA/venv/logFiles/%(program_name)s/worker.log ; stdout log path, NONE for none; default AUTO
stderr_logfile= /home/HBA/venv/logFiles/%(program_name)s/worker.log ; stderr log path, NONE for none; default AUTO
autostart=true ; start at supervisord start (default: true)
autorestart=true ; whether/when to restart (default: unexpected)
startsecs=0
stopwaitsecs=1000
killasgroup=true
My celery log file gives me:
**[2014-08-07 19:46:40,584: ERROR/MainProcess] Process 'Worker-4' pid:12284 exited with 'signal -1'
[2014-08-07 19:46:40,584: ERROR/MainProcess] Process 'Worker-3' pid:4432 exited with 'signal -1'
[2014-08-07 19:46:40,584: ERROR/MainProcess] Process 'Worker-2' pid:9120 exited with 'signal -1'
[2014-08-07 19:46:40,584: ERROR/MainProcess] Process 'Worker-1' pid:6280 exited with 'signal -1'**
C:\Python27\lib\site-packages\celery\apps\worker.py:161: CDeprecationWarning:
Starting from version 3.2 Celery will refuse to accept pickle by default.
The pickle serializer is a security concern as it may give attackers
the ability to execute any command. It's important to secure
your broker from unauthorized access when using pickle, so we think
that enabling pickle should require a deliberate action and not be
the default choice.
If you depend on pickle then you should set a setting to disable this
warning and to be sure that everything will continue working
when you upgrade to Celery 3.2::
CELERY_ACCEPT_CONTENT = ['pickle', 'json', 'msgpack', 'yaml']
You must only enable the serializers that you will actually use.
warnings.warn(CDeprecationWarning(W_PICKLE_DEPRECATED))
[2014-08-07 19:47:08,822: WARNING/MainProcess] C:\Python27\lib\site-packages\celery\apps\worker.py:161: CDeprecationWarning:
Starting from version 3.2 Celery will refuse to accept pickle by default.
The pickle serializer is a security concern as it may give attackers
the ability to execute any command. It's important to secure
your broker from unauthorized access when using pickle, so we think
that enabling pickle should require a deliberate action and not be
the default choice.
If you depend on pickle then you should set a setting to disable this
warning and to be sure that everything will continue working
when you upgrade to Celery 3.2::
CELERY_ACCEPT_CONTENT = ['pickle', 'json', 'msgpack', 'yaml']
You must only enable the serializers that you will actually use.
warnings.warn(CDeprecationWarning(W_PICKLE_DEPRECATED))
**[2014-08-07 19:47:08,944: INFO/MainProcess] Connected to amqp://guest:**#127.0.0.1:5672//
[2014-08-07 19:47:08,954: INFO/MainProcess] mingle: searching for neighbors
[2014-08-07 19:47:09,963: INFO/MainProcess] mingle: all alone**
C:\Python27\lib\site-packages\celery\fixups\django.py:236: UserWarning: Using settings.DEBUG leads to a memory leak, never use this setting in production environments!
warnings.warn('Using settings.DEBUG leads to a memory leak, never '
[2014-08-07 19:47:09,982: WARNING/MainProcess] C:\Python27\lib\site-packages\celery\fixups\django.py:236: UserWarning: Using settings.DEBUG leads to a memory leak, never use this setting in production environments!
warnings.warn('Using settings.DEBUG leads to a memory leak, never '
[2014-08-07 19:47:09,982: WARNING/MainProcess] celery#CORONADO ready.
I solved my issue using the following command: /home/HBA/venv/Scripts/celery worker -A runLogProject --loglevel=INFO
My biggest issue was an unfamiliarity with virtual environments. I needed to make sure the files were in the correct folders within the venv.