Related
I have a problem with communication with the workers created in my dynamicSupervisor. after start all my workers i try make a call to one by pid and is generated one error (always).
At the beginning of the child I keep the pid with ets. Then when I want to make a call to this child I get the pid from the ets by an identifier. So far so good.
The problem is when I do the following:
GenServer.call(
pid,
{:action_project, %{project_id: project_id, pid: :erlang.pid_to_list(pid)}}
)
returns following error:
[error] GenServer #PID<0.606.0> terminating
** (RuntimeError) attempted to call GenServer #PID<0.606.0> but no handle_call/3 clause was provided
(backercamp) lib/gen_server.ex:693: MyApplication.ProjectWorker.handle_call/3
(stdlib) gen_server.erl:661: :gen_server.try_handle_call/4
(stdlib) gen_server.erl:690: :gen_server.handle_msg/6
(stdlib) proc_lib.erl:249: :proc_lib.init_p_do_apply/3
Last message (from #PID<0.680.0>): {:action_project, %{pid: '<0.606.0>', project_id: "23"}}
State: %{pid: '<0.606.0>', project_id: "23"}
Client #PID<0.680.0> is alive
(stdlib) gen.erl:169: :gen.do_call/4
(elixir) lib/gen_server.ex:921: GenServer.call/3
What's the problem with this call?
DynamicSupervisor code:
defmodule MyApplication.Supervisor do
use DynamicSupervisor
alias MyApplication.Management
def start_link(arg) do
DynamicSupervisor.start_link(__MODULE__, arg, name: __MODULE__)
end
def init(arg) do
DynamicSupervisor.init(arg)
end
def start_project_worker(project_id) do
spec = {MyApplication.ProjectWorker, %{project_id: project_id}}
DynamicSupervisor.start_child(__MODULE__, spec)
end
def start_all_project_workers() do
Enum.each(Management.list_all_projects(), fn %{id: project_id} ->
IO.puts("Started for project Id [#{project_id}]")
# 1s between workers
:timer.sleep(1000)
start_project_worker("#{project_id}")
end)
end
def action_project(pid, project_id) do
GenServer.call(
pid,
{:action_project, %{project_id: project_id, pid: :erlang.pid_to_list(pid)}}
)
end
end
Worker code:
defmodule MyApplication.ProjectWorker do
use GenServer, restart: :transient
alias MyApplication.Settings
alias MyApplication.Management
def start_link(state) do
GenServer.start_link(__MODULE__, state)
end
def init(state) do
schedule_action_project(Settings.get_frequency_ms())
state = Map.put(state, :pid, :erlang.pid_to_list(self()))
persist_state(state)
{:ok, state}
end
def handle_info(:schedule_action_project, %{project_id: _project_id, pid: _pid} = state) do
action_by_state(state)
{:noreply, state}
end
def handle_call({:action_project}, %{project_id: _project_id, pid: _pid} = state) do
case action_by_state(state) do
true ->
terminate(state)
false ->
{:reply, state, state}
end
end
defp persist_state(state) do
IO.puts(" :: Add project_id [#{state.project_id}] and pid #{state.pid}")
:ets.insert_new(:project_backup, {state.project_id, state.pid})
end
defp delete_persist_state(project_id) do
IO.puts(" :: Delete project_id [#{project_id}]")
:ets.delete(:project_backup, project_id)
end
defp schedule_action_project(time) do
IO.puts(" :: Schedule_action_project [#{time}]")
Process.send_after(self(), :schedule_action_project, time)
end
defp terminate(%{project_id: project_id, pid: _pid} = state) do
IO.puts(
" :: Stop processed, everything is done! project_id [#{state.project_id}] and pid #{
state.pid
}"
)
delete_persist_state(project_id)
{:stop, :normal, state}
end
defp action_by_state(%{project_id: _project_id, pid: _pid} = state) do
action_by_project(Management.get_project!(state.project_id))
end
defp action_by_project(%Project{} = project) do
#do something in project
end
end
The issue is within a handler. You messed all the arguments up. state is an internal state of GenServer, you do not pass it through, you receive it in the callback. The following signature will match if your state is a map having keys project_id and pid and you call it like GenServer.call(pid, {:action_project}).
def handle_call({:action_project}, %{project_id: _project_id, pid: _pid} = state)
You should change this handler to match your caller:
def handle_call(
{:action_project, %{project_id: _project_id, pid: _pid}},
_from,
state
)
Also note that handle_call callback receives three parameters (unlike handle_cast, the second one being the source of the message.
Sidenote: handle_info callback has the wrong signature as well.
When trying to start my process from a DynamicSupervisor im getting the following error:
{:error,
{:EXIT,
{:badarg,
[
{:erlang, :apply,
[
BfgEngine.MarketService,
:start_link,
{{BfgEngine.MarketService, :start_link, ["1111"]}, :permanent, 5000,
:worker, [BfgEngine.MarketService]}
], []},
{:supervisor, :do_start_child_i, 3, [file: 'supervisor.erl', line: 379]},
{:supervisor, :handle_call, 3, [file: 'supervisor.erl', line: 404]},
{:gen_server, :try_handle_call, 4, [file: 'gen_server.erl', line: 661]},
{:gen_server, :handle_msg, 6, [file: 'gen_server.erl', line: 690]},
{:proc_lib, :init_p_do_apply, 3, [file: 'proc_lib.erl', line: 249]}
]}}}
The code im using is:
def start_market(market_id) do
spec = {MarketService, market_id}
DynamicSupervisor.start_child(__MODULE__, spec)
end
However it is not clear to me what is going wrong. What argument to which function is it thats not correct? How do I break down and read the given error message?
Update:
This is the init method of my supervisor:
#impl true
def init(initial_arg) do
DynamicSupervisor.init(
strategy: :one_for_one,
extra_arguments: [initial_arg]
)
end
Update 2:
This is the start_link of market_service:
def start_link(market_id) when is_bitstring(market_id) do
GenServer.start_link(__MODULE__, market_id, name: via_tuple(market_id))
end
Im using the default child_spec im getting from GenServer
Update 3:
Changing to:
def start_market(market_id) do
spec = {MarketService, market_id: market_id}
DynamicSupervisor.start_child(__MODULE__, spec)
end
Gives:
{:error,
{:undef,
[
{BfgEngine.MarketService, :start_link, [[], [market_id: "222"]], []},
{DynamicSupervisor, :start_child, 3,
[file: 'lib/dynamic_supervisor.ex', line: 654]},
{DynamicSupervisor, :handle_start_child, 2,
[file: 'lib/dynamic_supervisor.ex', line: 640]},
{:gen_server, :try_handle_call, 4, [file: 'gen_server.erl', line: 661]},
{:gen_server, :handle_msg, 6, [file: 'gen_server.erl', line: 690]},
{:proc_lib, :init_p_do_apply, 3, [file: 'proc_lib.erl', line: 249]}
]}}
Changing to:
def start_market(market_id) do
spec = {MarketService, :market_id, market_id}
DynamicSupervisor.start_child(__MODULE__, spec)
end
Gives:
** (ArgumentError) supervisors expect each child to be one of:
* a module
* a {module, arg} tuple
* a child specification as a map with at least the :id and :start fields
* or a tuple with 6 elements generated by Supervisor.Spec (deprecated)
Got: {BfgEngine.MarketService, :market_id, "222"}
(elixir) lib/supervisor.ex:657: Supervisor.init_child/1
(elixir) lib/supervisor.ex:744: Supervisor.child_spec/2
(elixir) lib/dynamic_supervisor.ex:304: DynamicSupervisor.start_child/2
You got badarg exception to function erlang:apply/3 when there are three arguments BfgEngine.MarketService, :start_link and {{BfgEngine.MarketService, :start_link, ["1111"]}, :permanent, 5000, :worker, [BfgEngine.MarketService]} and it happen in function supervisor:do_start_child_i/3.
The arguments to the function erlang:apply/3 should be MFA a.k.a Module, Function, Arguments. {{BfgEngine.MarketService, :start_link, ["1111"]}, :permanent, 5000, :worker, [BfgEngine.MarketService]} is not Arguments because it obviously is not a list of arguments. From your code, I can guess the error is the content of variable spec. You should provide some proplist or map. I don't know, you should read the documentation of DynamicSupervisor more carefully.
The error message
As of understanding the error message thrown by Elixir, you can refer to the official Erlang documentation. The section about errors and exceptions from Learn You Some Erlang For Great Good can help. The answer of #Hynek -Pichi Vychodil is also accurate.
Your specific problem
As #Milan Jaric mentioned, your error would come from:
def start_market(market_id) do
spec = {MarketService, market_id}
DynamicSupervisor.start_child(__MODULE__, spec)
end
But not only! DynamicSupervisor.start_child(__MODULE__, spec) is calling MarketService.start_link/1!
Your problem lies in the combination of this function from the DynamicSupervisor module, and the way you parse values in MarketService.start_link/1:
def start_link(market_id) when is_bitstring(market_id) do
GenServer.start_link(__MODULE__, market_id, name: via_tuple(market_id))
end
Indeed, this code should work, if you also have implemented MarketService.init/1 correctly. I am not able to reproduce the error. Are you sure market_id is really a bitstring?
Personally, I had based my code on the official documentation:
defmodule MySupervisor do
use DynamicSupervisor
def start_link(init_arg) do
DynamicSupervisor.start_link(__MODULE__, init_arg, name: __MODULE__)
end
def start_child(foo, bar, baz) do
# If MyWorker is not using the new child specs, we need to pass a map:
# spec = %{id: MyWorker, start: {MyWorker, :start_link, [foo, bar, baz]}}
spec = {MyWorker, foo: foo, bar: bar, baz: baz}
DynamicSupervisor.start_child(__MODULE__, spec)
end
#impl true
def init(init_arg) do
DynamicSupervisor.init(
strategy: :one_for_one,
extra_arguments: [init_arg]
)
end
end
As you can see, they suggest to use a Keyword list here:
spec = {MyWorker, foo: foo, bar: bar, baz: baz}
It works only if you've implemented MyWorker.start_link/1 as follows:
def start_link(args) do
foo = Keyword.fetch!(args, :foo)
bar = Keyword.fetch!(args, :bar)
baz = Keyword.fetch!(args, :baz)
Genserver.start_link(__MODULE__, {foo, bar, baz}, [])
def init({foo, bar, baz}) do
# do something...
state = {foo, bar, baz}
{:ok, state}
In your case, if you change start_market/1 to:
def start_market(market_id) do
spec = {MarketService, market_id: market_id}
DynamicSupervisor.start_child(__MODULE__, spec)
end
It will not work because this MarketService.start_link/1 will fail:
def start_link(market_id) when is_bitstring(market_id) do
GenServer.start_link(__MODULE__, market_id, name: via_tuple(market_id))
end
market_id here is not a bitstring but a Keyword list. So you got to modify MarketService.start_link/1 function to:
def start_link(args) when is_list(args) do
market_id = Keyword.fetch!(args, :market_id)
GenServer.start_link(__MODULE__, market_id, name: via_tuple(market_id))
end
And write a MarketService.init/1 as follows:
def init(market_id) do
# do something... Let's keep it simple for the example:
state = market_id
{:ok, state}
end
Helpful resources
José Valim's answer on a similar issue posted on Elixir Forum
Official module-based DynamicSupervisor documentation
A well-written series of articles about GenServer and Supervisors in Elixir...
...and its corresponding source code
It is hard to tell from posted code, but could you try changing start_market to:
def start_market(market_id) do
spec = {MarketService, :market_id, market_id}
DynamicSupervisor.start_child(__MODULE__, spec)
end
UPDATE (Below are two options):
def start_market(market_id) do
spec = &{
id: MarketService,
start: {MarketService, start_link, [market_id]},
type: :worker
}
DynamicSupervisor.start_child(__MODULE__, spec)
end
OR
def start_market(market_id) do
spec = {MarketService, [market_id]}
DynamicSupervisor.start_child(__MODULE__, spec)
end
I am using RestClient to make a network call in the ruby class. I am getting a SocketError whenever I am not connected to the internet. I have added a rescue block to catch the exception still I am not able to do so.
the error message is:
SocketError (Failed to open TCP connection to api.something.com:443 (getaddrinfo: Name or service not known))
module MyProject
class Client
def get_object(url, params={})
response = RestClient.get(url, {params: params})
rescue SocketError => e
puts "In Socket errror"
rescue => e
puts (e.class.inspect)
end
end
end
The broad rescue gets called and print SocketError, but why the previous rescue SocketError is not triggered!
Do you see something that I am missing?
There are a couple of exceptions you'll need to rescue in case you want to fail gracefully.
require 'rest-client'
def get_object(url, params={})
response = RestClient.get(url, {params: params})
rescue RestClient::ResourceNotFound => e
p e.class
rescue SocketError => e
p e.class
rescue Errno::ECONNREFUSED => e
p e.class
end
get_object('https://www.google.com/missing')
get_object('https://dasdkajsdlaadsasd.com')
get_object('dasdkajsdlaadsasd.com')
get_object('invalid')
get_object('127.0.0.1')
Because you can have problems regarding the uri being a 404 destination, or be a domain that doesn't existing, an IP, and so on.
As you can see above, you can have different scenarios that you may encounter when dealing with connecting to a remote uri. The code above rescue from the most common scenarios.
The first one the URL is missing, which is handled by RestClient itself.
The the following three below are invalid domains, which will fail with SocketError (basically a DNS error in this case).
Finally in the last call we try to connect to an IP that has no server running on it - therefore it throws an ERRNO::ECONNREFUSED
I have a problem with a long-time consuming migration, which I desired to run in the parallel (it can be runned in the parallel). Actually migration is about taking all records in the database and implement time- and resource- consuming operations on each of them.
Sometimes individual record migration hanging out, so I give 10 minutes to finish. If migration isn't finished, I want it to gracefully shut down without any exception (see below)
I'm also using poolboy erlang package to parallelize implementation since migration consumes not only the time, but resources too. The problem is that I don't know how to handle error when timeout happened and code is going to break. My supervision tree is:
defmodule MyReelty.Repo.Migrations.MoveVideosFromVimeoToB2 do
use Ecto.Migration
alias MyReelty.Repo
alias MyReelty.Repo.Migrations.MoveVideosFromVimeoToB2.Migrator
# parallel nature of migration force us to disable transaction
#disable_ddl_transaction true
#migrator_waiting_time 10 * 60 * 1000 # timeout
#poolboy_waiting_time #migrator_waiting_time + 10 * 1000 # give a time for graceful shutdown
#pool_name :migrator
#pool_size 3
#pool_config [
{ :name, { :local, #pool_name }},
{ :worker_module, Migrator },
{ :size, #pool_size },
{ :max_overflow, 0 },
{ :strategy, :fifo }
]
def up do
children = [
:poolboy.child_spec(#pool_name, #pool_config)
]
opts = [strategy: :one_for_one, name: MyReelty.Supervisor]
Supervisor.start_link(children, opts)
rows = Review |> Repo.all
IO.puts "Total amount of reviews is: #{length(rows)}"
parallel_migrations(rows)
end
def parallel_migrations(rows) do
Enum.map(rows, fn(row) ->
pooled_migration(#pool_name, row)
end)
end
def pooled_migration(pool, x) do
:poolboy.transaction(
pool,
(fn(pid) -> Migrator.move(pid, { x, #migrator_waiting_time }) end),
#poolboy_waiting_time
)
end
defmodule Migrator do
alias MyReelty.Repo
alias MyReelty.Review
use GenServer
def start_link(_) do
GenServer.start_link(__MODULE__, nil, [])
end
def move(server, { params, waiting_time }) do
GenServer.call(server, { :move, params }, waiting_time)
end
def handle_call({ :move, result }, _from, state) do
big_time_and_resource_consuming_task_here
{:reply, %{}, state}
end
end
end
The problem if migration of some record in the database takes more than 10 mins I have this kind of exception:
20:18:16.917 [error] Task #PID<0.282.0> started from #PID<0.70.0> terminating
** (stop) exited in: GenServer.call(#PID<0.278.0>, {:move, [2, "/videos/164064419", "w 35th st Springfield United States Illinois 60020"]}, 60000)
** (EXIT) time out
(elixir) lib/gen_server.ex:604: GenServer.call/3
(poolboy) src/poolboy.erl:76: :poolboy.transaction/3
(elixir) lib/task/supervised.ex:94: Task.Supervised.do_apply/2
(elixir) lib/task/supervised.ex:45: Task.Supervised.reply/5
(stdlib) proc_lib.erl:247: :proc_lib.init_p_do_apply/3
Function: #Function<5.53617785/0 in MyReelty.Repo.Migrations.MoveVideosFromVimeoToB2.parallel_migrations/1>
Args: []
20:18:16.918 [error] GenServer MyReelty.Repo terminating
** (stop) exited in: GenServer.call(#PID<0.278.0>, {:move, [2, "/videos/164064419", "w 35th st Springfield United States Illinois 60020"]}, 60000)
** (EXIT) time out
Last message: {:EXIT, #PID<0.70.0>, {:timeout, {GenServer, :call, [#PID<0.278.0>, {:move, [2, "/videos/164064419", "w 35th st Springfield United States Illinois 60020"]}, 60000]}}}
State: {:state, {:local, MyReelty.Repo}, :one_for_one, [{:child, #PID<0.231.0>, DBConnection.Poolboy, {:poolboy, :start_link, [[name: {:local, MyReelty.Repo.Pool}, strategy: :fifo, size: 1, max_overflow: 0, worker_module: DBConnection.Poolboy.Worker], {Postgrex.Protocol, [types: true, username: "adik", types: true, name: MyReelty.Repo.Pool, otp_app: :my_reelty, repo: MyReelty.Repo, adapter: Ecto.Adapters.Postgres, database: "my_reelty_dev", hostname: "localhost", extensions: [{Geo.PostGIS.Extension, [library: Geo]}, {Ecto.Adapters.Postgres.DateTime, []}, {Postgrex.Extensions.JSON, [library: Poison]}], pool_size: 1, pool_timeout: 5000, timeout: 15000, adapter: Ecto.Adapters.Postgres, database: "my_dev", hostname: "localhost", pool_size: 10, pool: DBConnection.Poolboy, port: 5432]}]}, :permanent, 5000, :worker, [:poolboy]}], :undefined, 3, 5, [], 0, Ecto.Repo.Supervisor, {MyReelty.Repo, :my_reelty, Ecto.Adapters.Postgres, [otp_app: :my_reelty, repo: MyReelty.Repo, adapter: Ecto.Adapters.Postgres, database: "my_reelty_dev", hostname: "localhost", extensions: [{Geo.PostGIS.Extension, [library: Geo]}], pool_size: 1]}}
I tried to insert terminate/2 or handle_info/2 to Migrator and play with it, but I even haven't reached this functions to be invoked. How can I handle timeouts and prevent them to break my migration?
UPDATED
I used #johlo's hint, but I still getting time out. My function is:
def init(_) do
Process.flag(:trap_exit, true)
{:ok, %{}}
end
When the Migrator.move/2 (i.e. the GenServer.call) function times out it will crash the entire MoveVideosFromVimeoToB2 process since that's the actual process that makes the GenServer call.
The solution here is to catch the timeout in the anonymous function in pooled_migration, something like (I'm not very familiar with Elixir syntax, so it might not compile, but you should get the idea) :
def pooled_migration(pool, x) do
:poolboy.transaction(
pool,
(fn(pid) ->
try do
Migrator.move(pid, { x, #migrator_waiting_time })
catch
:exit, reason ->
# Ignore error, log it or something else
:ok
end
end),
#poolboy_waiting_time
)
end
It's not the Migrator process that times out, it's the GenServer call to the Migrator that does and we need to try-catch that.
Also note that the Migrator process isn't killed it is still running, see the timeouts section in the GenServer call documentation.
UPDATE:
As #asiniy mentions in the comments the #poolboy_waiting_time should be set to :infinity so the poolboy.transaction function doesn't throw a timeout error when waiting for a free Migrator worker process. Since the Migrator will exit eventually this is safe.
The need to send and receive SMS messages via Smpp, Register Number, received systemID and password, but fails to connect. Connected to the project gem 'ruby-smpp', to use the example of the getaway this, it only changed the values systemID and password.
in the logs:
<- (BindTransceiver) len = 37 cmd = 9 status = 0 seq = 1364360797 (<systemID><password>)
Hex dump follows:
<- 00000000: 0000 0025 0000 0009 0000 0000 5152 7e5d | ...% ........ QR ~]
<- 00000010: 3531 3639 3030 0068 4649 6b4b 7d7d 7a00 | <systemID>.<password>.
<- 00000020: 0034 0001 00 | .4 ...
Starting enquire link timer (with 10s interval)
Delegate: transceiver unbound
in the console:
Connecting to SMSC ...
MT: Disconnected. Reconnecting in 5 seconds ..
MT: Disconnected. Reconnecting in 5 seconds ..
MT: Disconnected. Reconnecting in 5 seconds ..
MT: Disconnected. Reconnecting in 5 seconds ..
MT: Disconnected. Reconnecting in 5 seconds ..
Tell me, please, I do not, maybe in the config to something else to add or change? And smpp connection, as I understand, it works only with a specific IP-address, but the logs on the server and on the local machine are the same
class Gateway
include KeyboardHandler
# MT id counter.
##mt_id = 0
# expose SMPP transceiver's send_mt method
def self.send_mt(*args)
##mt_id += 1
##tx.send_mt(##mt_id, *args)
end
def logger
Smpp::Base.logger
end
def start(config)
# The transceiver sends MT messages to the SMSC. It needs a storage with Hash-like
# semantics to map SMSC message IDs to your own message IDs.
pdr_storage = {}
# Run EventMachine in loop so we can reconnect when the SMSC drops our connection.
puts "Connecting to SMSC..."
loop do
EventMachine::run do
##tx = EventMachine::connect(
config[:host],
config[:port],
Smpp::Transceiver,
config,
self # delegate that will receive callbacks on MOs and DRs and other events
)
print "MT: "
$stdout.flush
# Start consuming MT messages (in this case, from the console)
# Normally, you'd hook this up to a message queue such as Starling
# or ActiveMQ via STOMP.
EventMachine::open_keyboard(KeyboardHandler)
end
puts "Disconnected. Reconnecting in 5 seconds.."
sleep 5
end
end
# ruby-smpp delegate methods
def mo_received(transceiver, pdu)
logger.info "Delegate: mo_received: from #{pdu.source_addr} to #{pdu.destination_addr}: #{pdu.short_message}"
end
def delivery_report_received(transceiver, pdu)
logger.info "Delegate: delivery_report_received: ref #{pdu.msg_reference} stat #{pdu.stat}"
end
def message_accepted(transceiver, mt_message_id, pdu)
logger.info "Delegate: message_accepted: id #{mt_message_id} smsc ref id: #{pdu.message_id}"
end
def message_rejected(transceiver, mt_message_id, pdu)
logger.info "Delegate: message_rejected: id #{mt_message_id} smsc ref id: #{pdu.message_id}"
end
def bound(transceiver)
logger.info "Delegate: transceiver bound"
end
def unbound(transceiver)
logger.info "Delegate: transceiver unbound"
EventMachine::stop_event_loop
end
end
module KeyboardHandler
include EventMachine::Protocols::LineText2
def receive_line(data)
sender, receiver, *body_parts = data.split
unless sender && receiver && body_parts.size > 0
puts "Syntax: <sender> <receiver> <message body>"
else
body = body_parts.join(' ')
puts "Sending MT from #{sender} to #{receiver}: #{body}"
SampleGateway.send_mt(sender, receiver, body)
end
prompt
end
def prompt
print "MT: "
$stdout.flush
end
end
/initializers
require 'eventmachine'
require 'smpp'
LOGFILE = Rails.root + "log/sms_gateway.log"
Smpp::Base.logger = Logger.new(LOGFILE)
/script
puts "Starting SMS Gateway. Please check the log at #{LOGFILE}"
config = {
:host => '127.0.0.1',
:port => 6000,
:system_id => <SystemID>,
:password => <Password>,
:system_type => '', # default given according to SMPP 3.4 Spec
:interface_version => 52,
:source_ton => 0,
:source_npi => 1,
:destination_ton => 1,
:destination_npi => 1,
:source_address_range => '',
:destination_address_range => '',
:enquire_link_delay_secs => 10
}
gw = Gateway.new
gw.start(config)
file from the script / run through the rails runner
Problem solved. Initially were incorrectly specified host and port, as smpp-server was not on the same machine as the RoR-application. As for encoding, sending in Russian layout should be
message = text.encode ("UTF-16BE"). force_encoding ("BINARY")
Gateway.send_mt (sender, receiver, message, data_coding: 8)
To receive a form in the right
message = pdu.short_message.force_encoding ('UTF-16BE'). encode ('UTF-8')