I have a pylons based webapp and i'd love to use celery + rabbitmq for some time taking tasks. I've taken a look at the celery-pylons project but I haven't succeeded in using it.
My main problem with celery is: where do i put the celeryconfig.py file or is there any other way to specify the celery options eg. BROKER_HOST and the like, from within a pylons app (In the same way one can put the options in the django settings.py file when using django-celery).
Basically, i investigated 2 options: using celery as a standalone project and using celery-pylons, both without much success.. :(
Thanks in advance for your help.
I am doing this currently, although I've not updated celery in some time. I'm still on 2.0.0 I think.
What I did was to create a celery_app directory within my pylons application. (so in same directory as data, controllers, etc.)
In that directory are my celeryconfig.py, tasks.py, and pylons_tasks.py.
pylons_tasks.py is just a file that initializes the pylons application so I can load Pylons models and such into the celery tasks.py file. So it does the pylons init and then imports tasks.py.
The celeryconfig is then set to use myapp.celery_app.pylons_tasks as the CELERY_IMPORTS value.
CELERY_IMPORTS = ("myapp.celery_app.pylons_tasks", )
Hope that helps some.
The tightest integration with pylons is to build a custom loader into paste commands. This is what celery-pylons does. Check out my fork of celery-pylons https://bitbucket.org/dougtabuchi/celery-pylons/src which should work with the latest celery and pylons 1.0.
To get the celeryd side working you need to add the correct options in your ini file and then call paster celeryd development.ini
For the webapp side you just need to import celerypylons in environment.py Then you will be able to import and use your tasks from anywhere in your project.
A good pylons project to look at that uses celery is https://rhodecode.org/rhodecode/files/tip/
Related
I'm trying to figure out how to properly setup an ejabberd project that allows for easy compilation of custom beam files- so far, we've been using an existing project that is cumbersome to manage, and uses erlide as the IDE.
I would like to set up the project in a way that I can use a more helpful IDE like vscode, and somehow streamline the compiling and copying of the beam files and updating the module on the server.
Writing code in Elixir is fine as well- I just want the project to be set up in a way that is dev friendly.
Apologies if the question is too broad, but I'm not exactly sure how else to best phrase it. If you feel like I'm missing something in my current flow, please let me know, as I've basically inherited this project. If there are any clarifications required, let me know as well.
Thanks.
easy compilation of custom beam files
somehow streamline the compiling and copying of the beam files and updating the module on the server.
If the task is about compiling and loading additional modules, a running ejabberd node can compile, load and start additional modules in runtime, see
https://docs.ejabberd.im/developer/extending-ejabberd/modules/#ejabberd-contrib
Usually the modules come from
https://github.com/processone/ejabberd-contrib
but you can tell ejabberd to download other modules from other git repositories, or you can copy modules source code and tell ejabberd to install them. And those modules can be written in Erlang or Elixir. Full example: https://docs.ejabberd.im/developer/extending-ejabberd/elixir/#elixir-module-in-ejabberd-contrib
Basically:
you write the module in your development machine, test it...
when happy with the source code, copy mod_whatever.erl to the production machine, $HOME/.ejabberd-modules/sources/mod_whatever as explained in the example mentioned earlier
run ejabberdctl module_install mod_whatever
In step 2, instead of copying the source code yourself, you can have a git repository just for your module, tell ejabberd the module's git URL, similarly to https://github.com/processone/ejabberd-contrib/tree/master/extra
BTW, for step 3, starting in ejabberd 22.10, there's a page in ejabberd webadmin to install and uninstall those modules (copying the files requires manual administration of course).
I would like to set up the project in a way that I can use a more helpful IDE like vscode
What a coincidence, these days I'm playing with VSCode variants (VSCode, VSCodium, Coder's code-server and Github Codespaces) and how to develop ejabberd using them. This is useful for step 1 that I mentioned earlier (write module and test it). If you are interested in ejabberd + VSCode, tell me.
My end goal: I want to fetch data from a retail site on an hourly schedule to see if a specific product is back in stock or not.
I tried using xpath in python to scrape the site myself, but I'm not too familiar, and why reinvent the wheel if someone built a scraper already? In this case, Diggernaut has a github repo.
https://github.com/Diggernaut/configs/tree/master/bananarepublic.gap.com
I'm using the above github repo to try and run a pre-existing web scraper on the banana republic retail site. All that's included in the folder is a config.yml file. I don't even know where to start to try and run it... I am not familiar with using .yml files at all, barely know my way around a terminal (I can do basic "ls" and "cd" and "brew install", otherwise, no idea).
Help! I have docker and git installed (not that I know how to use docker). I have a Mac version 10.13.6 (High Sierra).
I'm not sure why you're looking at using Docker for this, as the config.yml is designed for use on Diggernaut.com and not as part of a docker container deployment. In fact, there is no docker container for Diggernaut that exists as far as I can see.
On the main Github config page for Diggernaut they list the following instructions:
All configs can be used with Diggernaut service to retrieve products information.
You need to create free account at Diggernaut
Login to your account
Create a project with any name and description you want
Get into your new project by clicking it and create new digger with any name
Then you will see 3 options suggested to you, you need to use one where you will use meta-language
Config editor will open and you can simply copy and paste config code and click on save button.
Switch mode for digger from Debug to Active and then run your digger.
Wait for completion.
Download data.
Schedule your runs if required.
I'm new to Docker. I'm trying to use it to develop CraftCMS. I found https://hub.docker.com/r/blackpepper/craftcms which works well, but I'd really like to add BrowserSync to it (so I need NodeJS in the container as well?).
I really have no idea where to begin. I've been creating Dockerfiles and docker-compose.yml files and just throwing random stuff at it... for instance in my Dockerfile I put:
FROM blackpepper/craftcms
FROM node
RUN npm install -g browser-sync
RUN npx browser-sync start --files=templates/index.twig,web/assets/style.css --proxy=http://localhost:8080
# the proxy url is where I can actually view the craftcms site in my browser
I'd really appreciate any guidance, especially if you have experience doing something like this.
After having struggled myself for years with setting up Node.js workflows including Browsersync and Gulp (or other node modules), I created an alternative that is muuch simpler to use and maintain.
I just depend on 3 binaries: inotifywait, xdotool and sassc.
If you want to know more, I have created a video showcasing the functionality.
You can check it here: https://youtu.be/tMW-xQZ6c1U
Hope that helps.
Swagger 3 ? or Open API ..? 3? not sure what they're calling it. There doesn't seem to be a good or easy setup.
I'm looking to add it to my java project
I don't mind manually filling out the config for my controllers and in fact do not want it dynamically config'd
and I don't want a lot of extra fluff with it.
Their github hosts a number of different files and I'm not sure what I do and don't need.
The setup at:
https://github.com/swagger-api/swagger-ui/blob/master/docs/development/setting-up.md
I've run through the setup commands and started locally which returns 2 js files, 1 css, and the swagger.json.
Does anyone know if there's a gradle or maven import / line that I can just plop into my build and get the code I need?
Has anyone setup swagger 3 with just the bare minimum to host the ui page with their controller on a java/springboot app?
any help welcome.
git clone https://github.com/swagger-api/swagger-ui.git
cd swagger-ui
npm install
npm run dev
I have been looking for a simple solution to importScript a full bundle created by webpack into a service worker. I have a few issues:
When I bundle a single file with webpack I can importScript this into my service worker. However, if I "require(import)" a file inside the file to be importScripted then the service worker complains that it does not know what export means. How do I fix this?
What format should my entry files be commonjs, umd...? Since importScript seems to just import the global functions of the file I don't even know if webpack is the right solution. Is it? Other people seems to have used it.
Can I import any node_module component I want just like on normal client side bundling? Are there limitations?
WHERE IS THE DOCUMENTATION??? Maybe I am misunderstanding something...
thanks in advance
That's not how service workers work.
importScripts is designed to import service worker code, not your application code.
What you really want is auto-generated service worker, based on your bundle. You can use this webpack plugin that uses excellent sw-precache under the hood.