From Sebesta's Concepts of Programming Languages, I know that the timeline of a program is generally considered to be:
language design time
language implementation time
compile time
load time
link time
run time
In chapter five it mentions three more times: declaration time, declaration-elaboration time, and elaboration time. All of these occur during runtime. I know this is a slightly different concept as it deals with the allocation of memory cells rather than overall program execution, but am unclear on how the general timeline for storage allocation works/fits in and can't seem to find anything in the book about it.
Are there additional times for storage allocation?
Which order do they go in?
Or are all three the same thing?
Related
I'm using an sqlite db to populate an ios app with data about irregular verbs -- it's a language learning app.
The db has 3 tables, verb categories, verbs, and verb elements. The last has one entry per verb termination, so 50-odd per verb. The total db is about 2,500 entries, each one has 3-5 short string fields.
My question is whether the best thing to do from a performance point of view is:
(1) - write structs to model the data, each with appropriate functions, and load the db into memory in the form of structs when the app initialises, OR
(2) - write functions to query the sqlite db on the fly for each new view
My guess is that (1) is much better for a small db of this size, but I thought I would consult your collective wisdom as I'm a novice. Thanks in advance for any advice.
Btw, this is a follow up question to one I asked about how to get data into an ios app:
What's the best practice for storing and importing (non-user) data for an iOS app in swift/swiftUI/Xcode?
Both solutions should be sufficient for your case.
When it comes to performance this is question on what you are measuring.
Quick loading time
Quick operations in background
Smooth user interface and interactions
Energy consumtions
Memory consumptions
...
For instance when increasing memory consumption you have to understand that your app will more likely be terminated when in background. If user decides to take off 5 minutes and opens a game which loads immense amount of memory your application will more likely be terminated to get that memory for another application.
But again. Both solutions should be fine for the size you are describing here. Over time if this increases you may need to reconsider.
Also there is an option 3 where you can just hardcode these values directly into structures so you even lose initial load time. Just make a script or something that transforms your files with strings to source code.
Have you thought about CoreData? I use it for a small DB and it works fine. There is of course the learning curve of CoreData, but once you get over that, it does a lot of the work for you and has some integrations with SwiftUI as well. I am not saying it is more or less performant in any way, just that once over the learning curve, it is pretty easy to use and of course optimized by the Apple CoreData team.
I'm working on a crowd simulator. The idea is people walking around a city in 2D. Think gray rectangles for the buildings and colored dots for the people. Now I want these people to be programmable by other people, without giving them access to the core back end.
I also don't want them to be able to use anything other than the methods I provide for them. Meaning no file access, internet access, RNG, nothing.
They will receive get events like "You have just been instructed to go to X" or "You have arrived at P" and such.
The script should then allow them to do things like move_forward or how_many_people_are_in_front_of me and such.
Now I have found out that Lua and python are both thousands of times slower than compiled languages (I figured it would be in order of magnitude of 10s times slower), which is way to slow for my simulation.
So heres my question: Is there a programming language that is FOSS, allows me to restrict system access (sandboxing) the entire language to limit the amount of information the script has by only allowing it to use my provided functions, that is reasonably fast, something like <10x slower than Java, where I can send events to objects inside that language with which I can load in new Classes/Objects on the fly.
Don't you think that if there was a scripting language faster than lua and python, then it'd be talked about at least as much as they are?
The speed of a scripting language is rather vague term. Scripting languages essentially are converted to a series of calls to functions written in fast compiled languages. But the functions are usually written to be general with lots of checks and fail-safes, rather than to be fast. For some problems, not a lot of redundant actions stacks up and the script translation results in essentially same machine code as the compiled program would have. For other problems, a person, knowledgeable about the language, might coerce it to translate to essentially same machine code. For other problems the price of convenience stay forever with the script.
If you look at the timings of benchmark tasks, you'll find that there's no consistent winner across them. For one task the language is fastest, for the other it is way behind.
It would make sense to gauge language speed at your task by looking at similar tasks in benchmarks. So, which of those problem maps the closest to yours? My guess would be: none.
Now, onto the question of user programs inside your program.
That's how script languages came to existence in the first place. You can read up on why such a language may be slow for example in SICP.
If you evaluate what you expect people to write in their programs, you might decide, that you don't need to give them whole programming language. Then you may give them a simple set of instructions they can use to describe a few branching decisions and value lookups. Then your own very performant program will construct an object that encompasses the described logic. This tric is described here and there.
However if you keep adding more and more complex commands for users to invoke, you'll just end up inventing your own language. At that point you'll likely wish you'd went with Lua from the very beginning.
That being said, I don't think the snippet below will run significantly different in compiled code, your own interpreter object, or any embedded scripting language:
if event = "You have just been instructed to go to X":
set_front_of_me(X) # call your function
n = how_many_people_are_in_front_of_me() #call to your function
if n > 3:
move_to_side() #call to function provided by you
else:
move_forward() #call to function provided by you
Now, if the users would need to do complex computer-sciency stuff, solve np-problems, do machine learning or other matrix multiplications, then yes, that would be slow, provided someone would actually trouble themselves with implementing that.
If you get to that point, it seem that there are at least some possibilities to sandbox the compiled dlls (at least in some languages). Or you could do compilation of users' code yourself to control the functionality they invoke and then plug it in as a library.
This post on Erlang scalability says there's an overhead for every call, cast or message to a gen_server. How much overhead is it, and what is it for?
The cost that is being referenced is the cost of a (relatively blind) function call to an external module. This happens because everything in the gen_* abstractions are callbacks to externally defined functions (the functions you write in your callback module), and not function calls that can be optimized by the compiler within a single module. A part of that cost is the resolution of the call (finding the right code to execute -- the same reason each "dot" in.a.long.function.or.method.call in Python or Java raise the cost of resolution) and another part of the cost is the actual call itself.
BUT
This is not something you can calculate as a simple quantity and then multiply by to get a meaningful answer regarding the cost of operations across your system.
There are too many variables, points of constraint, and unexpectedly cheap elements in a massively concurrent system like Erlang where the hardest parts of concurrency are abstracted away (scheduling related issues) and the most expensive elements of parallel processing are made extremely cheap (context switching, process spawn/kill and process:memory ratio).
The only way to really know anything about a massively concurrent system, which by its very nature will exhibit emergent behavior, is to write one and measure it in actual operation. You could write exactly the same program in pure Erlang once and then again as an OTP application using gen_* abstractions and measure the difference in performance that way -- but the benchmark numbers would only mean anything to that particular program and probably only mean anything under that particular load profile.
All this taken in mind... the numbers that really matter when we start splitting hairs in the Erlang world are the reduction budget costs the scheduler takes into account. Lukas Larsson at Erlang Solutions put out a video a while back about the scheduler and details the way these costs impact the system, what they are, and how to tweak the values under certain circumstances (Understanding the Erlang Scheduler). Aside from external resources (iops delay, network problems, NIF madness, etc.) that have nothing to do with Erlang/OTP the overwhelming factor is the behavior of the scheduler, not the "cost of a function call".
In all cases, though, the only way to really know is to write a prototype that represents the basic behavior you expect in your actual system and test it.
I have been searching the web for some simple world clock program that will run in the background without the use of internet connection. Instead I found this site using Google search: http://wwp.greenwichmeantime.com/info/global.htm
Now I would like some opinions on which language would fit best to the program I would like. Basically it will look like a simple 2 column table. Column 1 will be the time zone and column 2 as current time according its respective zones. It will just continuously run in the background while not consuming too much memory. The local time of the computer will be used as the base.
Again this will be an OFFLINE world time zone, so a desktop-application.
Features may include these as well but not necessary:
The time will be at 24 hour format to make it easier to read (well I find it easier and might be necessary).
Transparency setting.
Always on top.
I have learned some basic programming languages but don't know how to make this project in actuality since I haven't touched on the "visual" parts of the languages. I've seen some applications built on java but I haven't learnt it yet. I know basic python and basic C++.
Here's what I think it will look like: http://imgur.com/8ZdisWX
Just make it longer vertically so every time zone will fit.
First some notes. The page you link to show 24 timezones. This is for all practical use a lie. There are isn't 24 time zones, there are hundreds. 408, to be specific, of which several have more than one name, so there are 548 different names. http://en.wikipedia.org/wiki/List_of_tz_database_time_zones If you are on Windows, you might want to use Windows internal registry of time zones. It has fewer, but still hundreds of timezones.
Listing all of those is not practical. You will need a way for the user to select what time zones he wants to display. Many operating systems time/date widgets allow this already.
If you still want to do this application, then you can do this in any language whatsoever, more or less. Since you know basic Python I'd recommend you do it in Python, using pytz as the time zone library. As for GUI support, Kivy looks cool, and is very portable, even supporting iOS and Android.
Also note that time zones change often, so that you probably need to update your application a couple of times per year, just to get the latest time zone definitions. You can get around that in various more or less complicated ways. But I'd leave that for overcourse at the moment.
In general use, should I bet on memory efficiency or processor efficiency?
In the end, I know that must be according to software/hardware specs. but I think there's a general rule when there's no boundaries.
Example 01 (memory efficiency):
int n=0;
if(n < getRndNumber())
n = getRndNumber();
Example 02 (processor efficiency):
int n=0, aux=0;
aux = getRndNumber();
if(n < aux)
n = aux;
They're just simple examples and wrote them in order to show what I mean. Better examples will be well received.
Thanks in advance.
I'm going to wheel out the universal performance question trump card and say "neither, bet on correctness".
Write your code in the clearest possible way, set specific measurable performance goals, measure the performance of your software, profile it to find the bottlenecks, and then if necessary optimise knowing whether processor or memory is your problem.
(As if to make a case in point, your 'simple examples' have different behaviour assuming getRndNumber() does not return a constant value. If you'd written it in the simplest way, something like n = max(0, getRndNumber()) then it may be less efficient but it would be more readable and more likely to be correct.)
Edit:
To answer Dervin's criticism below, I should probably state why I believe there is no general answer to this question.
A good example is taking a random sample from a sequence. For sequences small enough to be copied into another contiguous memory block, a partial Fisher-Yates shuffle which favours computational efficiency is the fastest approach. However, for very large sequences where insufficient memory is available to allocate, something like reservoir sampling that favours memory efficiency must be used; this will be an order of magnitude slower.
So what is the general case here? For sampling a sequence should you favour CPU or memory efficiency? You simply cannot tell without knowing things like the average and maximum sizes of the sequences, the amount of physical and virtual memory in the machine, the likely number of concurrent samples being taken, the CPU and memory requirements of the other code running on the machine, and even things like whether the application itself needs to favour speed or reliability. And even if you do know all that, then you're still only guessing, you don't really know which one to favour.
Therefore the only reasonable thing to do is implement the code in a manner favouring clarity and maintainability (taking factors you know into account, and assuming that clarity is not at the expense of gross inefficiency), measure it in a real-life situation to see whether it is causing a problem and what the problem is, and then if so alter it. Most of the time you will not have to change the code as it will not be a bottleneck. The net result of this approach is that you will have a clear and maintainable codebase overall, with the small parts that particularly need to be CPU and/or memory efficient optimised to be so.
You think one is unrelated to the other? Why do you think that? Here are two examples where you'll find often unconsidered bottlenecks.
Example 1
You design a DB related software system and find that I/O is slowing you down as you read in one of the tables. Instead of allowing multiple queries resulting in multiple I/O operations you ingest the entire table first. Now all rows of the table are in memory and the only limitation should be the CPU. Patting yourself on the back you wonder why your program becomes hideously slow on memory poor computers. Oh dear, you've forgotten about virtual memory, swapping, and such.
Example 2
You write a program where your methods create many small objects but posses O(1), O(log) or at the worst O(n) speed. You've optimized for speed but see that your application takes a long time to run. Curiously, you profile to discover what the culprit could be. To your chagrin you discover that all those small objects adds up fast. Your code is being held back by the GC.
You have to decide based on the particular application, usage etc. In your above example, both memory and processor usage is trivial, so not a good example.
A better example might be the use of history tables in chess search. This method caches previously searched positions in the game tree in case they are re-searched in other branches of the game tree or on the next move.
However, it does cost space to store them, and space also requires time. If you use up too much memory you might end up using virtual memory which will be slow.
Another example might be caching in a database server. Clearly it is faster to access a cached result from main memory, but then again it would not be a good idea to keep loading and freeing from memory data that is unlikely to be re-used.
In other words, you can't generalize. You can't even make a decision based on the code - sometimes the decision has to be made in the context of likely data and usage patterns.
In the past 10 years. main memory has increased in speed hardly at all, while processors have continued to race ahead. There is no reason to believe this is going to change.
Edit: Incidently, in your example, aux will most likely end up in a register and never make it to memory at all.
Without context I think optimising for anything other than readability and flexibilty
So, the only general rule I could agree with is "Optmise for readability, while bearing in mind the possibility that at some point in the future you may have to optimise for either memory or processor efficiency in the future".
Sorry it isn't quite as catchy as you would like...
In your example, version 2 is clearly better, even though version 1 is prettier to me, since as others have pointed out, calling getRndNumber() multiple times requires more knowledge of getRndNumber() to follow.
It's also worth considering the scope of the operation you are looking to optimize; if the operation is time sensitive, say part of a web request or GUI update, it might be better to err on the side of completing it faster than saving memory.
Processor efficiency. Memory is egregiously slow compared to your processor. See this link for more details.
Although, in your example, the two would likely be optimized to be equivalent by the compiler.