I am trying to scrape a website using playwright and host it on Azure Functions. However, playwright requires Chrome drivers which I can't seem to install into Azure Functions.
Unsure how to do this step in a pipeline:
pip install playwright
playwright install
Previously, I tried to do it in an azure pipeline:
- script: |
python -m venv ./azfunc/.python_packages
source ./azfunc/.python_packages/bin/activate
python -m pip install --upgrade pip keyring artifacts-keyring
pip install -r ./azfunc/requirements.txt
python -m playwright install
displayName: 'Install dependencies'
However, I get a webkit" browser was not found.Please complete Playwright installation via running"python -m playwright install
It seems like the Azure Function is not finding where it was installed. Any suggestions? Would using Docker be a good tactic?
Looks like azure functions support headless chromium with consumption based app service plan.
Here is the blog to know more about the running headless chromium in azure functions with playwright
over this github issue ,it was suggested to use the azure functions on docker container
please file a support ticket and MS Q&A here
Related
I have an Asp web api application using .net 5. The application is using Playwright to generate pdfs. It works locally on iis without any issues.
When I run it using docker windows containers on the following image:
FROM mcr.microsoft.com/dotnet/aspnet:5.0 AS base
it complains about missing Chromium. I tried installing playwright on that image using multiple ways but failed. The image doesn't have dotnet .sdk installed nor powershell. In my bin folder I do have playwright installation files at:
bin\Debug\net5.0\.playwright\node\win32_x64
If I run from there:
playwright.cmd /install
It downloads and installs browser binaries but in my Asp app I'm now getting error about missing dependencies:
PlaywrightException: Host system is missing dependencies!
Full list of missing libraries: dwrite.dll uiautomationcore.dll
winspool.drv dxgi.dll
I tried similar approach on Linux containers but with similar results. The approach from the Playwright tutorial using "dotnet tool" command doesn't work, because there is no dotnet sdk.
What is the best way to make Playwright work on that asp image?
Installed aws-cdk using pip on windows10. After restart, when i run cdk --version on cmd prompt, it says 'cdk' is not recognized as an internal or external command,
operable program or batch file.
aws --version
aws-cli/2.0.30 Python/3.7.7 Windows/10 botocore/2.0.0dev34
python --version
Python 3.8.1
Do I need to install NodeJS as well?
Please help me setup cdk on my windows10 laptop....
Based on the official documentation you need to have Node.js installed.
And I'll quote from the documentation
All CDK developers need to install Node.js 10.3.0 or later, even those working in languages other than TypeScript or JavaScript.
Also..
Install the AWS CDK Toolkit globally using the following Node Package Manager command.
npm install -g aws-cdk
After that you should be able to run cdk --version without any issues
I'm new to Docker and I'm not very experienced in SSL/certificates etc.
I'm working on a web application that lets the user log in to JIRA via JIRA API to do things. This works on my computer and I don't get any SSL errors. However, when I run it in a Docker container I get
ssl.SSLError: [SSL: WRONG_SIGNATURE_TYPE] wrong signature type error.
My friend ran the exact same Dockerfile in his computer and created a container and it works which is confusing.
I checked the requests library version on my computer and compared it to the one in Docker container but they are the same. What could be the problem? Thank you
Note: I use Windows
Faced similar problem.
Find the solution here
Just to sum up:
There is missing dependencies inside container which your own system already has. You should install them inside docker.
Idk how but pyopenssl library should be install to
So, you need to add:
RUN apt-get update \
&& apt-get install openssl \
&& apt-get install ca-certificates
In your Dockerfile
And add:
pyopenssl==19.1.0
To your requirements.txt
If you don't use requirements.txt just add:
RUN pip install pyopenssl
To your Dockerfile
Hope it'll help
Similar issue happened to me - python tls requests were working fine on my host, but failed with WRONG_SIGNATURE_TYPE once I dockerized my script.
The issue seems to stem from where requests lib uses older TLS stacks with OpenSSL 1.1.1pre9.
I noticed rolling back to python:3.5.3 stopped the error, however for compatibility with newer versions the solution posted in github issue thread worked fine.
i.e. import the TLSAdapter, use it to setup a requests session, then start making requests.
I have a ruby on rails app running on a Google Cloud Platform VM running on the app engine flexible environment. It looks like it installs most of the software on the VM when I deploy the app with gcloud --project project-name preview app deploy I think it installs rails and other software from reading the temporary dockerfile it creates. It grabs the info for the dockerfile from the app.yaml file (I got this setup from following their tutorials).
This was working fine for me but now I need to install ImageMagick onto the server to manipulate images on the site. Normally you do this by running sudo apt-get install imagemagick from the project directory. When I SSH onto the VM I cant find the project directory so that doesn't work.
I have no idea how to get it to run sudo apt-get install imagemagick each time I make a new deploy to the site so it has the software on the new VM.
As you might be able to tell I'm not very good with the server side of things and want to know what I'm supposed to do to get new software onto the VM the right way so its always there like ruby and rails etc.. are each time I make a new deploy.
You should use a custom Dockerfile when you need additional configuration.
To install ImageMagics you have to set runtime: custom in your app.yaml, create a Dockerfile based on default one, and add following line:
RUN apt-get update && \
apt-get install imagemagick -y
I want to use xvfb to automate the process to create thumbnails images on urls from my site.
It is not clear where I should download a standalone xvfb application.
Where do I download the package?
How do I build the package?
Are there any other packages to create images of a rendered url on the linux server using php or shell scripting?
I am on Linux Red Hat Enterprise Linux Server release 5.5 (Tikanga)
After some search, I find it's part of the XFree86 implementation of X11. http://ftp.xfree86.org/pub/XFree86/4.8.0/binaries/Linux-x86_64-glibc23/
Update: the above link is dead. If you're using a standard linux OS, try search xvfb with system package manager (e.g. apt or yum)
sudo yum install -y xorg-x11-server-Xvfb
or
It's available on the Client, Workstation, and Server Optional Channels inside RHN: (Note the URLs here will prompt for your redhat support account)
https://rhn.redhat.com/network/software/packages/name_overview.pxt?package_name=xorg-x11-server-Xvfb&archIdList=&archLabelList=&search_subscribed_channels=yes