Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about a specific programming problem, a software algorithm, or software tools primarily used by programmers. If you believe the question would be on-topic on another Stack Exchange site, you can leave a comment to explain where the question may be able to be answered.
Closed 4 years ago.
Improve this question
tl;dr
I've currently got a PostGresQL database with about 10gb of data. This data is "archived" data- so it won't ever be changing, but I do need the data to be queryable/searchable/available for reading in the cheapest method possible for my Rails app.
Details:
I'm running a Digital Ocean server, but this is a no-profit project, so keeping costs low is essential. I'm currently using a low-end droplet 4 GB Memory / 40 GB Disk / SFO2 - Ubuntu 16.04.1 x64
Querying this data/loading the pages it's used on can take a significant amount of time occasionally. Some pages timeout because they take over a minute to load. (Given, those are very large pages, but still)
I've been looking at moving the database over to Amazon RedShift, but the base prices seem large- as they're aimed at MUCH larger projects than mine.
Is my best bet to try to put more and more into making the queries small and only rendering small bits at a time? Even basic pages have a long query time because the server is slowed down so much. Or is there a method similar to RedShift that will allow me to quickly query the data while also storing it externally for a reasonable price?
You can try Amazon S3 and Amazon Athena. S3 is a super simple storage where you can dump your data in text files and Athena is a service that provides SQL-like interface to data stored on S3. S3 is super cheap and Athena has per runtime cost. Since you said your data isn't going to change and is going to be queried rarely it's a good solution. Check this out: 9 Things to Consider When Choosing Amazon Athena
Related
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 6 years ago.
Improve this question
We are changing our current user side MySQL database. We have a website + mobile app for which users around the US query our database. the relevant data is is contained in three tables, and a join query against the three tables is needed to send the relevant results to the users.
The result sent back to the users are of small size (<6kb). If our objective is to go for low latency and throughput is a low priority, which of the two following databases would perform better:
MemSQL or AWS Aurora?
they both have the same starting cost for hardware (~$0.28/hr). We are only considering those two databases at this stage so that we can continue our "MySQL" in-house knowledge.
I like that i can outsource the DB headache to Aurora. But surely MemSQL's ability to read/write to memory is a lower latency solution?
Nothing beats in-memory for speed and this is what MemSQL is built for. It stores tables (in rowstore mode) in memory and uses a custom query engine to cache queries into an intermediate language so it can execute as fast as possible. Aurora is more like a classic disk-based MySQL instance but with lots of infrastructure changes and optimizations to make the most of Amazon's services.
Before deciding though, you need to figure out what "low-latency" means - is this within seconds or milliseconds?
MemSQL will be faster and most likely in milliseconds depending on your query. Aurora will be slower but can probably deliver sub-second, again depending on your query and the resources allocated and how the data is structured.
Without any more details, the answer is to judge by what your performance tolerance is and then experiment.
Closed. This question is off-topic. It is not currently accepting answers.
Want to improve this question? Update the question so it's on-topic for Stack Overflow.
Closed 10 years ago.
Improve this question
I know it sounds crazy but I'm sitting here with various phone on different networks and all the phones on the sprint network are failing to work correctly.
It looks as if the LESS style sheet is not being applied. Has any one ever ran into any thing like this?
Also just visited the LESS website figuring all there styles would be created with LESS and its doing the same thing. Failing to load/apply the LESS.
The specific phones I have tried on the sprint network are two iphones and one android optimus V.
I would strongly suggest pre-processing your LESS file into CSS and serving that on your site.
It is considerably more efficient as even a medium sized less file can take hundreds of milliseconds to process during page load. It is also one less javascript file to download in production. Finally, it is a lot less processor overhead on mobile devices that need to not only parse the javascript, but then parse the less file as well. Some mobile devices don't have caching or local storage so there is a potential for them to be re-processing the less file every time the page loads.
Either use the lessc compiler (requires Node.js) or SimpLESS to pre-compile your css files.
You could try to see if it's blocking the download of the less javascript or the stylesheet itself by loading their URLs directly.
For lesscss.org that would be http://lesscss.org/less/main.less and http://lesscss.org/js/less.js
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about a specific programming problem, a software algorithm, or software tools primarily used by programmers. If you believe the question would be on-topic on another Stack Exchange site, you can leave a comment to explain where the question may be able to be answered.
Closed 2 years ago.
Improve this question
I want to get started with my first Ruby on Rails application.
It will pull an image, and some text about the image display both, and have a small box to write some text about the image, which will then be written to a database once submitted.
It's unlikely to have any more than 5 concurrent users, as it's a personal project.
What's the lowest VPS spec needed to run ruby on rails? Would it be possible on 64mb (128 burst) ram or could I go even lower?
The lowest I'd advocate is a 512MB system. The Ruby on Rails stack can be 50-100MB alone unless you're very careful about pruning off extras. This is an inconsequential amount of memory on a modern system, though, where 4096MB is common even in the VPS world.
Linode offers a $19.95 plan for the basic 512MB system which, while not the cheapest around, is very affordable even for personal projects. There are less expensive providers, but their quality of service may vary considerably.
If you're using Passenger then even a 512MB machine can run several lightly loaded sites.
Instead of running your own VPS you might want to use Heroku which doesn't allocate memory to customers directly but instead shares of CPU time they call "Dynos" that are somewhat more abstract than a VPS.
Younger Joseph. You should've learn to use git and started using Heroku. Heroku doesn't publicize it clearly on the website but they offer a free plan.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 1 year ago.
Improve this question
Which graph database should I use when dealing with a couple of thousand nodes and a couple of thousands relationships? Are these big numbers for any database or not? Which graph database is the fastest at read operations (assuming all data is loaded once at the beggining).
I had a look at neo4j and its visualization tool. Will I be able to have such a visualization tool in my application?
The questions you'll need to ask and answer for a graph database are similar to any other database. How much data? In memory or persistent? How will you interface with it? Embedded or a server process? Distributed or localized? Licensing?
A couple of thousand nodes and relationships is small for a graph database and most any graph database solution will work. For most people Neo4j is a fine choice, but there are some caveats. First, the licensing of Neo4j can be problematic in many situations. Secondly, the visualizer is part of the Neo4j server process - which means you're going to have another server process running. If you're concerned about the licensing you may want to check out OrientDB, which is under the Apache license, and thus very flexible.
From the sounds of it, you have a fairly small system and may be able to get by with using TinkerGraph, an in-memory graph database from Marko Rodriguez and the Tinkerpop hackers. It has the option to persist your data to a file if needed, is amazingly lightweight, and, like Neo4j and OrientDB, supports all the graph tools from the Tinkerpop stack, including the Jung Ouplemntation, which can give you the visualizations you desire.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about a specific programming problem, a software algorithm, or software tools primarily used by programmers. If you believe the question would be on-topic on another Stack Exchange site, you can leave a comment to explain where the question may be able to be answered.
Closed 5 years ago.
Improve this question
I have a program that reads about a million of rows and group the rows; the client computer is not stressed at all, no more than 5% cpu usage and the network card is used at about 10% or less.
If in the same client machine I run four copies of the program the use grow at the same rate, with the four programs running, I get about 20% cpu usage and about 40% network usage. That makes me think that I can improve the performance using threads to read the information from the database. But I don't want to introduce this complexity if a configuration change could do the same.
Client: Windows 7, CSDK 3.50.TC7
Server: AIX 5.3, IBM Informix Dynamic Server Version 11.50.FC3
There are a few tweaks you can try, most notably setting the fetch buffer size. The environment variable FET_BUF_SIZE can be set to a value such as 32767. This may help you get closer to saturating the client and the network.
Multiple threads sharing a single connection will not help. Multiple threads using multiple connections might help - they'd each be running a separate query, of course.
If the client program is grouping the rows, we have to ask "why?". It is generally best to leave the server (DBMS) to do that. That said, if the server is compute bound and the client PC is wallowing in idle cycles, it may make sense to do the grunt work on the client instead of the server. Just make sure you minimize the data to be relayed over the network.