Use SO_REUSEPORT with Phoenix - network-programming

I am interested in a zero-downtime deployment system that does not use Elixir/Erlang hot upgrades (due to the complexity of data migrations while the code is running).
I have heard that I may be able to use the SO_REUSEPORT option when binding the server to the adapter, such that I can run two instances of the same application bound to the the same address and port. My intent is to deploy version 2 on the same server as a running version 1, start version 2, and then gracefully stop version 1, which should allow incoming connections to naturally begin connecting exclusively to version 2.
Regardless of whether this works exactly as I plan - my intent is to test this configuration knowing that it behaves differently on different OSes - I would like to know the specific steps necessary to configure Phoenix to do this, as this seems to be a lower-level configuration within :gen_tcp.
If, alternatively, there is a way to configure the OS or Erlang VM to make all connections with this option enabled by default, that would be even better.

You should specify raw SO_REUSEPORT flag for a socket in the format {:raw, protocol, option_num, value_bin}
gen_tcp option/raw and pass it to the underlying transport.
Please note, that flags are different for mac/linux. In your config.exs:
so_reuseport =
case :os.type() do
{:unix, :linux} -> {:raw, 1, 15, <<1::32-native>>}
{:unix, :darwin} -> {:raw, 0xffff, 0x0200, <<1::32-native>>}
end
config :yourapp, YourApp.Endpoint,
http: [port: {:system, "PORT"}, transport_options: [socket_opts: [so_reuseport]]]
Tested on Phoenix 1.4.9, but I guess older versions should be ok too.
Here are the corresponding docs for used options.
http Phoenix.Endpoint — Phoenix
v1.4.9
transport_options Plug.Cowboy — PlugCowboy
v2.1.0

Related

How to make container installation behave like host machine installation

I'm working with the following:
Docker for Windows v20.10.11
Docker running in Windows container mode
mcr.microsoft.com/windows:1903 base image
Proprietary application installed on top of this base image
Each year we create a Docker image with the latest version of our company's software. However this year's version behaves differently. Host machine installation runs fine. Containerized installation fails to run in certain situations. I can start the application as a simple EXE, for example using the Docker run command. The app will start and show up in "tasklist". However I can't start the app via the COM API, which is a critical requirement. The problem appears to be COM related. Normally we can create COM objects for our software just like for any other application. For example, IE returns a COM object just fine:
Creating these objects for our application works outside containers. However inside the container, our latest installation gives this error:
Access permissions appear to be ok. I tried a couple tests to prove this. First I can install other software like MS Word into a container and create COM objects for that:
Second I tried retrieving + modifying the application's DACL in PowerShell.
Changing access masks or trustees can cause an Access Denied error:
This also appears to confirm the access permissions were Ok by default.
Next I made sure COM is aware of the application. This appears to be fine. I get the same result on host machine and container when running this PS script:
gci HKLM:\Software\Classes -ea 0| ? {$.PSChildName -match '^\w+.\w+$' -and
(gp "$($.PSPath)\CLSID" -ea 0)} | ft PSChildName
The application shows up just like any other. The details show up fine when querying by AppID. LocalServer32 points to the correct EXE:
Some other things I tried:
Querying registry keys. There are 7 keys created when installing our software. These appear identical on host machine install and container install.
Even though permissions appear fine, I still tried logging into the container as alternate users. For example "nt authority\system" is another virtual admin user. I also changed the password of the "builtin\administrator" user to enable logging in with that one. Lastly tried creating new users entirely and adding them to the Administrators user group. All these attempts had the same errors as "builtin\containeradministrator" (default user).
A minor check was ensuring CMD.exe / Powershell is running as x64:
Re-registering the DLLs associated with the installation using regsvr32.
Starting from different base images. https://learn.microsoft.com/en-us/virtualization/windowscontainers/manage-containers/container-base-images. The full Win Server base image behaves exactly the same way regarding errors. The smaller Win Server Core base image is even more problematic, as I can't even start the app's EXE manually using that base. Lastly I tried other tags of the full Windows base image such as 20H2 and 2004. Same result from those. Multiarch or x64 makes no difference.
Included the "Ogawa hack" which was historically needed to make MS Office apps function correctly with COM: https://stackoverflow.com/a/1680214/7991646. It could be necessary for other COM apps too, but didn't help with my specific installation.
Is there anything else I can do to diagnose or solve this COM issue?
There are several things to consider:
The Considerations for server-side Automation of Office article states the following:
Microsoft does not currently recommend, and does not support, Automation of Microsoft Office applications from any unattended, non-interactive client application or component (including ASP, ASP.NET, DCOM, and NT Services), because Office may exhibit unstable behavior and/or deadlock when Office is run in this environment.
If you are building a solution that runs in a server-side context, you should try to use components that have been made safe for unattended execution. Or, you should try to find alternatives that allow at least part of the code to run client-side. If you use an Office application from a server-side solution, the application will lack many of the necessary capabilities to run successfully. Additionally, you will be taking risks with the stability of your overall solution.
The When CoCreateInstance returns 0x80080005 (CO_E_SERVER_EXEC_FAILURE) page describes possible reasons.
If many COM+ applications run under different user accounts that are specified in the This User property, the computer cannot allocate memory to create a new desktop heap for the new user. Therefore, the process cannot start. See Error when you start many COM+ applications: Error code 80080005 -- server execution failed for more information.
Finally, you may find a similar thread here helpful, see Server execution failed (Exception from HRESULT: 0x80080005 (CO_E_SERVER_EXEC_FAILURE)).

Progress ABL How to Test for WEBSPEED in the PRE-PROCESSOR

I want to conditionally compile some blocks of code depending on type of client i'm running in. this is fine for batch and tty as i can use the {&BATCH-MODE} but how to test for when the code is being compiled in webspeed agent? eg. {&IF} not {&SOMETHING} EQ "YES" {&THEN}
{&ANALYSE-SUSPEND}
foo
bar
{&ANALYSE-RESUME}
{&ENDIF}
it would be helpful if this did not rely on defines auto generated by the architect in .w's etc but that would be a nice to have not essential.
Compile time isn't run time. If the program can be run different ways (as a part of a of webpage using webspeed, as a part of a batch and as a part of some other kind of client etc) you're most likely better of evaluating this in run time instead.
You can identify in what environment you're running:
SESSION:CLIENT-TYPE
This will identify your type of client.
DISPLAY SESSION:CLIENT-TYPE.
Type of client Attribute value
-------------------------------- -----------------------
ProVision standard ABL client 4GLCLIENT
WebClient WEBCLIENT
AppServer agent APPSERVER
WebSpeed agent WEBSPEED
Pacific Application Server agent MULTI-SESSION-AGENT
Other special-purpose clients Unknown value (?)
Documentation
Using VST
If you have at least one database connected
_Connect-ClientType tells you what kind of client this particular connection is:
Value Client
-------- ---------------------
ABL ABL client
SQLC SQL client
WTA Webspeed agent
APSV AppServer agent
SQFC SQL Federated client
Example:
FIND FIRST _myconnection NO-LOCK.
FIND FIRST _connect NO-LOCK WHERE _connect._connect-usr = _myconnection._MyConn-userid.
DISPLAY _connect._Connect-ClientType.
Based on OS
Perhaps you run different OS:es?
DISPLAY OPSYS.
Other ways
There's a number of other ways of doing this, including perhaps looking at PROPATH, Working directory etc.
Try to stick with a solution that won't change over the course of time because of Progress upgrades, new OS:es, new directory structures etc.
IMHO there is no such preprocessor variable out of the box.
But you could create your own include file and include that in the code that's relevant. You need two versions of that file, one says
&GLOBAL-DEFINE WebSpeed WebSpeed
and the other
&GLOBAL-DEFINE NoWebSpeed NoWebSpeed
And then configure your compile sessions so that they find exactly one of the files in propath.
But as you will agree, this is probably dangerous as the result will heavily rely on the proper PROPATH used during compilation. I'd rather attempt to use a runtime condition instead.
What are you trying to achieve in detail?
finally figured it out this morning {&webstream} and {&out} are not defined in in normal sessions so i can just test for that. runtime is not an issue in my case i just want to compile the code in all cases. in this shop dont ask me why but every single piece of code is session compiled. poor cpu but there u go. i could be defensive and add some logic with session:Client-type for bells and whistles you're right. if not can-do then boogie :)

Dartium and it's use of dart:io

I am building a feature-rich standalone application using dart. I picked dart since it seemed to allow not only the standard development approach for webapps but also to access system resources (such as nodejs does). One of my requirements is file io or database access (which again requires file io). Dartium however does not allow the use of the dart:io package (only in servermode is this library accessible). Can anybody think of a workaround, a change to the dart environment or the chromium environment to allow this ? Maybe a custom compiled dartVM in the browser environemnt.
Basically a way to use database connections in a dart standalone app. A REST wrapper is only an option if this code could get generated automatically.
I am not clear if such a thing is possible or if chromium will prevent any approach to access system resources.
Yes in standalone mode dart you can use files, and also TCP socket, and web socket.
So you can connect to databases. (for example to MySQL: https://github.com/jamesots/sqljocky)
In browser mode you can use built-in databases dart:indexed_db or dart:web_sql.
You can also use web-socket protocol
Maybe you can use websocket to connect with local or remote database.

Can I edit an iPad's host file?

I doubt this is possible without extensive jail-breaking, but is it at all possible to edit the iPad's (or any iOS device's) hosts file?
The previous answer is correct, but if the effect you are looking for is to redirect HTTP traffic for a domain to another IP there is a way.
Since it technically is not answering your question, I have asked and answered the question here:
How can I redirect HTTP requests made from an iPad?
No, you can't change iPad's host file(without jailbreak), but can workaround.
Here is my scenario:
Mac OS X, with IP 192.168.2.1, running a web app
iPad, the device you would like to test the web app
Charles (for Mac), enables HTTP proxy for your iPad
I am going to test the web app running in my Mac via iPad, but I can't access directly to it.
The solution works for me:
Firstly, make sure that your server and iPad are in the same local network.
Then, set up Charles proxy, in the menu "Proxy > Proxy Settings...", fill in Port(mostly 8888) and toggle Enable transparent HTTP proxying.
Setup proxy setting in iPad.
Now you can visit your web app in iPad.
Of course you can use other proxy tools like Squid or Varnish in Linux, or fiddler in Wondows.
No. Apps can only modify files within the documents directory, within their own sandbox. This is for security, and ease of installing/uninstalling. So you could only do this on a jailbroken device.
The easiest way to do this is to run an iPad simulator using XCode and then add an entry in the hosts file (/etc/hosts) on the host system to point to your test site.
I needed the same functionality, and doing jailbreak is no-no. One solution is to host yourself DNS server (MaraDNS), go to your wifi settings in ipad/phone, and add your custom DNS server there.
The whole process took me only 10 minutes, and it works!
1) Download MaraDNS
2) Run mkSecretTxt.exe as administrator
3) Modify mararc file, mine is:
ipv4_bind_addresses = "put your public IP Here"
timestamp_type = 2
random_seed_file = "secret.txt"
csv2 = {}
csv2["Simple.Example.com."] = "example.configuration"
Add file called "example.configuration" into the same folder where run_maradns.bat is.
4) Edit your example.configuration file:
Simple.Example.com. 10.10.13.13 ~
5) Disable all Firewalls (convenience)
6) Run file "run_maradns.bat"
7) There should be no errors.
8) Add your DNS server to list, as shown here: http://www.iphonehacks.com/2014/08/change-dns-iphone-ipad.html
9) Works!
Yes, you can edit the iPad hosts file, but you need to be jailbroken. Once you've done that, download Cydia (app market), and get iFile. The hosts file is located within "/etc/hosts".
I would imagine you could do it by setting up a transparent proxy, using something like charles and re-direct traffic that way
Workarond I use for development purposes:
Create your own proxy server (One option would be: Squid on Linux).
Set your hosts file with your domains.
Set the proxy server on the IPAD/IPHONE and you can use with your hosts.
I know it's been a while this has been posted, but with iOS 7.1, a few things have changed.
So far, if you are developing an App, you MUST have a valid SSL certificate recognized by Apple, otherwise you will get an error message on you iDevice. No more self-signed certificates. See here a list:
http://support.apple.com/kb/ht5012
Additionally, if you are here, it means that you are trying to make you iDevice resolve a name (to your https server), on a test or development environment.
Instead of using squid, which is a great application, you could simply run a very basic DNS server like dnsmasq. It will use your hosts file as a first line of name resolution, so, you can basically fool your iDevice there, saying that www.blah.com is 192.168.10.10.
The configuration file is as simple as 3 to 4 lines, and you can even configure its internal DHCP server if you want.
Here is mine:
listen-address=192.168.10.35
domain-needed
bogus-priv
no-dhcp-interface=eth0
local=/localnet/
Of course you have to configure networking on your iDevice to use that DNS (192.168.10.35 in my case), or just start using DHCP from that server anyway, after properly configured.
Additionally, if dnsmasq cannot resolve the name internally, it uses your regular DNS server (like 8.8.8.8) to resolve it for you. VERY simple, elegant, and solved my problems with iDevice App installation in-house.
By the way, solves many name resolution problems with regular macs (OS X) as well.
Now, my rant: bloody Apple. Making a device safe should not include castrating the operating system or the developers.
If you have the freedom to choose the hostname, then you can just add your host to a dynanmic DNS service, like dyndns.org. Then you can rely on the iPad's normal resolution mechanisms to resolve the address.
You can also make use of a proxy server on your iPhone or iPade via mobile internet (3G) by using a iPhone Mobile proxy generator:
http://iphonesettings.net/mobileproxygenerator.php
Just enter the apn of your carrier (with apn username/password if needed) and the proxy server you want to go through and tap Generate
Problem Restated: Bypassing DNS Resolution
The problem the OP is trying to solve is NOT hacking hosts files on iPads per se, but rather bypassing DNS Resolution of a specific published DNS record by creating a static, local IP:name mapping on their device.
Solution:
A better- and more scalable- way is to create the static IP:Name mapping that you'd create in the hosts file on the device and instead create it on the router and then point your DHCP addressed clients to that router as the primary source of DNS resolution as I document (with annotated screen shots) HERE.
Conclusion:
When testing a new site you need to check the display of it on multiple devices to ensure there's no funky display or usability issues. Bypassing DNS checking using static local IP:Name mappings would require each developer to hack the hosts files for each of their devices and then remember to unwind the changes in all their devices after testing.
And were it even possible to hack an iPad's hosts file as the OP enquired about, in many organizations the Developers IT assets will be locked-down and they won't they have administrative permissions to do such tinkering.
Better to make the static mapping to bypass the published DNS record in the router and then you can delete it in one place after testing is completed.
You need access to /private/etc/ so, no. you cant.
Best Answer: Simply add http or https in your browser, the IP address, colon and port number. Example: https://123.23.145.67:80

azure generating false wsdl address

I start my azure application, I browse to the svc file http://127.0.0.1:82/transaction.svc,
and I get the usual:
You have created a service.
To test this service, you will need to
create a client and use it to call the
service. You can do this using the
svcutil.exe tool from the command line
with the following syntax:
svcutil.exe
http://rzv-pc:5100/Transaction.svc?wsdl
If i go to http://rzv-pc:5100/Transaction.svc?wsdl the age is blank.
If I go to http://127.0.0.1:82/transaction.svc?wsdl, I get the wsdl but there are still references like
<xsd:import schemaLocation="http://rzv-pc:5100/Transaction.svc?xsd=xsd1"
I can't use svcutil on any of this links. What is the problem? I use Azure SDK 1.1.
See http://code.msdn.com/wcfazure (particularly the "known issues" page). The issue stems from the fact that there are two ports involved. One is the external port (that the load balancer listens on), which is port 82 in your case. The other is the internal port, that your code listens to on its VM in the cloud, which is port 5100 in your case.
Note that in SDK 1.3, you can specify the localPort attribute on your endpoints, which lets you choose what that internal port is. You can make it the same as the external port and thus avoid all strangeness.

Resources