We have a lua script which filters the records , returns the map. I have two questions
Does aerospike executes the lua script like an independent script (similar to 'lua ' ) on every query?
There is a need to read a file and cache it using a function -- I want this function to be called only once , how can it be achieved?
Aerospike executes Lua script in a sandboxed environment. The context is reset across calls. So, you cannot read a file and cache values which you can use during next invocation. If you need to pass some information to each call, consider passing them via arguments. Needless to say, its better to not pass huge data structures as arguments. The overhead of encoding/decoding them will be high.
Related
From what I've Googled, there are no global variables in Erlang?
Say I have function A (initialization code) which reads some info from a binary file into a few variables. I need to persist these variables for subsequent use in function B. Function B will be called many times whenever required.
What's the recommended practice for doing this?
Ih you are looping function B and there is no change of configuration, you could just pass the configuration arguments to function B.
If configuration could be changed or it is too much overhead, I usually store the configuration paramets in an ets table.
This is what I have also observed by other developers.
You can also check this short ets introduction by learnyousomeerlang.
function_B(Arg1, ConfigVars) ->
% do some stuff and modify Arg1
function_B(Arg1_Modified, ConfigVars).
I am writing an application which requires a very fast response time, I have some queries to redis which would require intersection and union of multiple sets .
An example would be
((A union B) intersection C)
However when I do it with java client, each query requires 1 more handshake thus increasing my response time.
I was wondering if there was a way that I could do it in a single handshake, Lua scripting looks like a good option but I'm not sure how it would work internally
You can also use pipeline to reduce the RTT.
With pipeline, you can send multiple commands to Redis at one time, and read all replies latter.
Redis lua script is a blocking method which does everything at once
inside the redis server.
so you will avoid network round trip for
sure which you needed.
Since it is blocking you should always be
aware of when to use it, if all those union and intersection takes a
5 sec in production environment then it is blocking for other
commands that are waiting to be executed. other commands may not get executed and comes out throwing an time out error which may cause things worse.
So alternately you can do
partial lua script calls for every 50 elements are so (derive an
optimal number by trying out with different numbers).
Also consider using sunion and sinter of multiple elements instead of using only 2 if it fits.
ie,
sunion set1 set2 set3...
instead of
sunionstore temp set1 set2
sunionstore temp temp set3 and so on.
Hope this helps.
Setup by running sadd a b c
When I execute this code against the set a
keystoclear1 has a single value of "b" in it.
keystoclear2 as both values in it.
local keystoclear = unpack(redis.call('smembers', KEYS[1]))
redis.call('sadd', 'keystoclear1', keystoclear)
redis.call('sadd', 'keystoclear2', unpack(redis.call('smembers', KEYS[1])))
I am by no means a lua expert, so I could just have some strange behavior here, but I would like to know what is causing it.
I tested this on both the windows and linux version of redis, with redis-cli and the stackexchange.redis client. Same behavior in all cases. This is a trivial example, I actually would like to store the results of the unpack because I need to perform several operations with it.
UPDATE: I understand the issue.
table.unpack() only returns the first element
Lua always adjusts the number of results from a function to the circumstances of the call. When we call a function as a statement, Lua discards all of its results. When we use a call as an expression, Lua keeps only the first result. We get all results only when the call is the last (or the only) expression in a list of expressions.
This case is slightly different from the one you referenced in your update. In this case unpack (may) return several elements, but you only store one and discard the rest. You can get other elements if you use local keytoclear1, keytoclear2 = ..., but it's much easier to store the table itself and unpack it as needed:
local keystoclear = redis.call('smembers', KEYS[1])
redis.call('sadd', 'keystoclear1', unpack(keystoclear))
As long as unpack is the last parameter, you'll get all the elements that are present in the table being unpacked.
How can I get current date / time in Lua embedded in Redis?
I need to have it in following format - YYYY-MM-DD, HH:MM:SS
Tried with os.date() but it does not recognize it.
Redis' Lua sandbox has only a handful of libraries, and os isn't one of these.
You can call the Redis TIME from Lua like so:
local t = redis.call('TIME')
However, you'll need to find a way to convert the epoch to the desired format and also note that it will stop you script from performing any writes (as it is a non-deterministic command).
Update: as of Redis v3.2, there is a new replication mode for scripts that is effect-based (rather than code-based). When using this mode you can actually call all the random, non-deterministic commands. More information is at EVAL's documentation page
This was already discussed in the comments, but the correct answer should have an answer:
The current time is non-deterministic i.e. it returns different values on repeated calls. This hurts replication. For this reason, the current time should be passed into your LUA script as a parameter.
I have a cluster app that uses a distributed Redis back-end, with dynamically generated Lua scripts dispatched to the redis instances. The Lua component scripts can get fairly complex and have a significant runtime, and I'd like to be able to profile them to find the hot spots.
SLOWLOG is useful for telling me that my scripts are slow, and exactly how slow they are, but that's not my problem. I know how slow they are, I'd like to figure out which parts of them are slow.
The redis EVAL docs are clear that redis does not export any timekeeping functions to lua, which makes it seem like this might be a lost cause.
So, short a custom fork of Redis, is there any way to tell which parts of my Lua script are slower than others?
EDIT
I took Doug's suggestion and used debug.sethook - here's the hook routine I inserted at the top of my script:
redis.call('del', 'line_sample_count')
local function profile()
local line = debug.getinfo(2)['currentline']
redis.call('zincrby', 'line_sample_count', 1, line)
end
debug.sethook(profile, '', 100)
Then, to see the hottest 10 lines of my script:
ZREVRANGE line_sample_count 0 9 WITHSCORES
If your scripts are processing bound (not I/O bound), then you may be able to use the debug.sethook function with a count hook:
The count hook: is called after the interpreter executes every
count instructions. (This event only happens while Lua is executing a
Lua function.)
You'll have to build a profiler based on the counts you receive in your callback.
The PepperfishProfiler would be a good place to start. It uses os.clock which you don't have, but you could just use hook counts for a very crude approximation.
This is also covered in PiL 23.3 – Profiles
In standard Lua C, you can't. It's not a built-in function - it only returns seconds. So, there are two options available: You either write your own Lua extension DLL to return the time in msec, or:
You can do a basic benchmark using a millisecond-resolution time. You can access the current millisecond time with LuaSocket. Though this adds a dependency to your project, it's an effective way to do trivial benchmarking.
require "socket"
t = socket.gettime();