Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 8 years ago.
Improve this question
Hi I am implementing Email Client Application. My Application is like web based Application. I have xxxxxx number of users Mail Credentials. I need to sync for them without using any looping concept. It should immediately reflect when something changes in Mail Server. IMAP IDLE will not helpful seems it is only giving New Email Arrival. I want to get Each and Every changes from Mailserver. Without using any looping, is there alternate way to do this. I need to do this in Generalized Approach. bcoz Something supported in Gmail but not supported in yahoo...
so generalized approach is appreciated.
The optimized way to avoid loops is called IDLE. That's why it was specified and implemented. If there had been a way to do what you want without IDLE, then IDLE would not have been necessary in the first place.
Your questions show a lack of familiarity with a protocol, and a weird entitlement to features that do not exist. I suspect very much you have bitten off way more than you can chew.
Read RFC 3501 cover to cover. This is the IMAP4Rev1 specification. This is all you can expect, in general, from general servers on the internet, and no more. And sometimes, not even that much*. IMAP4 is fundamentally a single-folder protocol. You can only get information and messages for one folder at a time. It is up to you to interrogate the server for what information you want; it is not obligated to tell you anything you did not specifically ask for.
Writing real, full-featured, broadly-compatible email clients is difficult. Start by writing a client that can synchronize one user and one folder, on baseline IMAP. Implement extensions to make this more efficient when available: IDLE, CONDSTORE, etc. Then move on to many folders, and many users.
Also understand that this dream of thousands of users is also going to run into serious logistical issues. No server on the internet is going to allow you to log in hundreds of times without having gotten an agreement ahead of time. You will cause your users extreme headaches when they get mysteriously locked out of their accounts for logging in from 'weird' places. In the case of google, this can be mitigated with OAuth, but that will only take you so far before they become interested in what you're doing.
Also, good reading is RFC 2683: IMAP Implementation Recommendations.
*: There are many servers that do not properly support SEARCH, several that do not send UIDNEXT on SELECT, Yahoo won't allow you to keep a persistent connection or give any updates at all without reconnecting, etc.
Related
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 9 years ago.
Improve this question
I've been a developer for 10 years (non iOS), and working for a large company have never created applications end to end. Just worked on very large applications, on pieces.
I'm starting to get into iOS for fun, and have an app in my head that I want to create. I've wireframed the entire thing using the iOS app 'interface'. Since then, I've begun to start coding. I have about 15 scenes in storyboard (the total app will probably be 100+), and right now I'm just using hardcoded 'fake' data.
However, I've recently begun to think that maybe I should be creating the database and some initial data there instead of using all this hardcoded fake data.
Does anyone have any suggestions and reasons why one way is better than another?
Should I create the back end before the front end? If I do, then each new scene I add I can work the real data in from the beginning instead of having to replace fake hard coded data.
Also, I know little about creating back ends. The application I'm creating is nothing like twitter, but for data access and for this example, let's say it is. It's main view of the application is something like twitter. The user can hit refresh and get many new data points ('tweets' in twitter) from the server. So the application could be very data intensive. Am I best off using something like Parse and paying for their services, or creating something in LAMP, or something else. I've worked with SQL and database a lot in my last 10 years and am very comfortable with that aspect of the back end.
Thoughts? Suggestions?
Thanks!
I'd say you have 3 options here :
Front-end first, back-end afterwards
Good thing is while developing your front-end, you may understand what's really relevant and what isn't. You probably won't do anything unnecessary on the back-end part. A bad thing though is that bad stuff may happen when you try to connect you back-end to your front-end, and involve some code refactoring on the front-end side, if you don't make sure they at least can work together.
Back-end first, front-end afterwards
You may here not really see where you're going while developing the back-end. You'll see (you may even know it already) that what you'll create for the client-side may not really be as it looked in your head.. You'll probably have to rework a lot on the back-end.
Front-and-back-end together
This is how I usually work. Start the front-end just as you did with hard-coded data, and start asap to work on the back-end. Move your boilerplate data on it, just so you can make sure they communicate well. Then, try to work on both simultaneously. That way, if you change your mind about something on one side, you won't have to redo much code on the other side.
Regarding the back-end solution, pretty much all I can say is that I used Parse.com services, and it's really good. In my case, I was not ready to create an entire back-end by myself. If you can, maybe you don't need them. But, (and it's a big one), Parse's SDK can take care of the whole communication between your back-end and your front-end. You don't have to manage network availability, caching stuff, and every thing you have to think about when you develop for a mobile OS. This is very nice.
Their free plan lets you run 1M queries every month, which is quite a lot. But if you want to go further and reduce the number of requests to Parse, you can combine your own back-end and theirs. It may not work for your specific case, but you can have the user access your server to check if new data is available, and only then query Parse. For example, for a news app, have the news on parse.com, store the most recent news date on your server, save the last update date on the client device, and before accessing parse, compare the dates with your server. If needed, query parse, if not, go to the cache (handled by parse's SDK). That way you can limit the number of queries and stay in the free plan.
You should probably try to estimate the number of queries you'll have per month and the monetary impact before choosing.
Just my own opinion :]
I would suggest you to add new features to your app with the smallest possible complexity. Like e. g. "The user can see a list of all registered users." - This example might not fit perfectly well for your case, but I hope you get the point: build one small thing at a time.
But for these small things: make the full trip front and back. Since it shouldn't take you too long to complete such a feature, it doesn't really matter if you complete the frontend or the backend first. So for this part: basically what #rdurand already said ;)
Regarding the backend I see two options:
Either you create some REST-Services yourself. The choice of technilogy should depend on what you know already. I am a big fan of JAX-RS, but if you don't already have some java experience you might have hard time with this.
Use some kind of SAAS-API. I've heard some good things about http://www.apiomat.com/, but never used it myself...
Good luck ;)
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 4 years ago.
Improve this question
This may be better suited to Server Fault, but it seems more of a programming challenge to me. I could be wrong.
I was thinking about how Facebook does what it does. It has over 500 million active users. How do they manage to serve all of those users? Is there one gigantic database holding a record for every single user so that whenever someone on logs in, authentication is checked against that central machine? I'm pretty ignorant about this topic but I can see that a solution like that is simple not scalable. There will come a point where that central server just can't handle everything.
Instead, say that central database is split up into 100 databases so that the load is split across all of them evenly. That must be what Facebook does, but how do they know which user record to store on what machine? Is there a record stored in every single machine and when you log in, a random user machine is used for authentication? That would mean every time someone registers or changes their password, the changes have to be propagated across all 100 servers.
One other solution comes to mind. Maybe they have some way of hashing a user's email address to a specific user database. Then all that would have to be known by the web servers is that hashing algorithm. But this solution brings up its own problem I think. What if you want to add more user database machines? Do you change the hashing algorithm to take into account 101 user databases instead of 100? Would you start moving user records around so the 101 user databases have the same number of user records? No, that seems ridiculous as well.
Anyways, as you can see I don't know too much about how to solve these problem. Does anyone have some recommended reading about this topic?
A good starting point might be to take a look at Cassandra (lecture notes), the distributed database that powers FB's inbox search.
Here's more about FB's nuts and bolts.
You might also find some gems in the FB developer news.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 4 years ago.
Improve this question
My company has this Huge Database that gets fed with (many) events from multiple sources, for monitoring and reporting purposes. So far, every new dashboard or graphic from the data is a new Rails app with extra tables in the Huge Database and full access to the database contents.
Lately, there has been an idea floating around of having external (as in, not our company but sister companies) clients to our data, and it has been decided we should expose a read-only RESTful API to consult our data.
My point is - should we use an API for our own projects too? Is it overkill to access a RESTful API, even for "local" projects, instead of direct access to the database? I think it would pay off in terms of unifying our team's access to the data - but is it worth the extra round-trip? And can a RESTful API keep up with the demands of running 20 or so queries per second and exposing the results via JSON?
Thanks for any input!
I think there's a lot to be said for consistency. If you're providing an API for your clients, it seems to me that by using the same API you'll understand it better wrt. supporting it for your clients, you'll be testing it regularly (beyond your regression tests), and you're sending a message that it's good enough for you to use, so it should be fine for your clients.
By hiding everything behind the API, you're at liberty to change the database representations and not have to change both API interface code (to present the data via the API) and the database access code in your in-house applications. You'd only change the former.
Finally, such performance questions can really only be addressed by trying it and measuring. Perhaps it's worth knocking together a prototype API system and studying it under load ?
I would definitely go down the API route. This presents an easy to maintain interface to ALL the applications that will talk to your application, including validation etc. Sure you can ensure database integrity with column restrictions and stored procedures, but why maintain that as well?
Don't forget - you can also cache the API calls in the file system, memory, or using memcached (or any other service). Where datasets have not changed (check with updated_at or etags) you can simply return cached versions for tremendous speed improvements. The addition of etags in a recent application I developed saw HTML load time go from 1.6 seconds to 60 ms.
Off topic: An idea I have been toying with is dynamically loading API versions depending on the request. Something like this would give you the ability to dramatically alter the API while maintaining backwards compatibility. Since the different versions are in separate files it would be simple to maintain them separately.
Also if you use the Api internally then you should be able to reduce the amount of code you are having to maintain as you will just be maintaining the API and not the API and your own internal methods for accessing the data.
I've been thinking about the same thing for a project I'm about to start, whether I should build my Rails app from the ground up as a client of the API or not. I agree with the advantages already mentioned here, which I'll recap and add to:
Better API design: You become a user of your API, so it will be a lot more polished when you decided to open it;
Database independence: with reduced coupling, you could later switch from an RDBMS to a Document Store without changing as much;
Comparable performance: Performance can be addressed with HTTP caching (although I'd like to see some numbers comparing both).
On top of that, you also get:
Better testability: your whole business logic is black-box testable with basic HTTP resquest/response. Headless browsers / Selenium become responsible only for application-specific behavior;
Front-end independence: you not only become free to change database representation, you become free to change your whole front-end, from vanilla Rails-with-HTML-and-page-reloads, to sprinkled-Ajax, to full-blown pure javascript (e.g. with GWT), all sharing the same back-end.
Criticism
One problem I originally saw with this approach was that it would make me lose all the amenities and flexibilities that ActiveRecord provides, with associations, named_scopes and all. But using the API through ActveResource brings a lot of the good stuff back, and it seems like you can also have named_scopes. Not sure about associations.
More Criticism, please
We've been all singing the glories of this approach but, even though an answer has already been picked, I'd like to hear from other people what possible problems this approach might bring, and why we shouldn't use it.
Closed. This question is off-topic. It is not currently accepting answers.
Want to improve this question? Update the question so it's on-topic for Stack Overflow.
Closed 9 years ago.
Improve this question
At a place I used to work they typical response to any problem was to blame the hardware or the users for not using the system perfectly. I had adopted the philosophy that it's my fault until I can prove otherwise prior to that job (and so far, at least 99 times out of 100 it's correct).
One of the last "unsolvable" problems when I was there was an abundance of database timeouts. After months of research, I still only had theories but couldn't prove any of them. One of my developers adamantly suggested replacing the network (every router, switch, access point) but couldn't provide any evidence that the network was the cause; it was, however, "obviously the cause" according to my manager (no development/IT experience) so he took over the problem. One caveat and Fog Creek plug: He couldn't account for the fact that the error reporting via FogBugz worked perfectly and to the same SQL Server as the rest of the data.
A couple, timeout-free months later, my manager boasted that he had fixed the timeouts ("Look, no timeouts!"). I had to hold back from grabbing a rock and saying "Look, no tigers!" but I did ask how he knew they would have occurred to which I got no response. The timeouts did return (and in greater numbers) a couple months later.
I'm pretty content with how I handled the situation but I'm curious how the SO crowd would have responded to letting a superior/colleague implement a solution you know (or are very sure) is wrong and will likely waste thousands of dollars?
Let them, but at the same time continue searching for the real cause.
A couple thousand dollars is money well spent if it keeps me from going against that kind of thinking (which is futile).
Well, if the problem is upper management, then I would do as you have done - lodge your complaint, then follow instructions. If it turns out they were right (it happens every now and then) then you look like a good employee despite your misgivings. If it turns out you were right, then they might be more willing to listen to you given that you allowed them their turn.
This is, of course, optimistic.
In the case of a colleague, take the problem up a level and consult your superiors for advice on how to approach the subject. Be fair to both your perspective and that of your colleague's, then follow the advice you're given.
Sometimes it's best to leave a manager be. If you think about his pressures and responsibilities he had to be seen to be doing something, rather than "nothing". After enough time "investigating" resolves into nothing to outside parties that need the timeouts to stop.
By taking an action he creates an opportunity to keep researching. The trick is to find a way to put your solutions in his context. Here is something we can do now, and here's what we can continue to do. For example, "We can replace the networking gear as a precautionary step, and then look at the version control logs to rule out that possibility."
This gives him something proactive so he can look productive up his chain while achieving the solution you want which will ultimately be successful.
In the long term you should look to work for someone who trusts your technical decisions implicitly, you can talk candidly with and who well help you help him navigate the politics in a way you both know what's going on. If you manager isn't that person, move.
Is this a big problem? Its not your job to save your company dollars other than that you would like your company to remain solvent so you get paid.
If its just one manager, he will be weeded out sooner or later, if your entire company culture is like this, maybe it would be time to move on.
In the mean time, see if you can see this from your manager's perspective.
I'd consider you're manager's intent to be a good thing. It's the people that don't want to bother that I find more difficult. It's just best to find a way to use that energy to be helpful.
One common problem for lots of people (occasionally myself), is that they flail around when trying to diagnose a problem. If you're wildly guessing at it, then particularity with modern computers, you have only the slimmest possibility of being right. Approaching this type of problem with that type of attitude, generally means that you'll never fix it.
The best way of handing complex debugging, is by divide and conquer. In this case, first think up a test, them implement it. Did that test act as expected? Depending on where you are with your tests, you are getting closer or farther away from the problem. The key, is that ALL of the tests must result in some concrete (objective) behavior. If the results are ambiguous, then the test is useless.
If you're getting a disconnect in part of the system, but some other part is not, then you have a huge amount of valuable information (it also shows it's not the network). What's the difference between the parts? Just start descending until you get somewhere ...
Getting back to your manager. Whenever I encounter that type of personality problem, I try to redirect the energy into something more useful. The desire is there, it just needs some help in getting shaped. If you can convince your manager to make sure the tests are concrete, then if they do enough of them, they'll produce enough information to correctly guess the bug. Sure, a more consistent approach might be faster, but why turn down some free assistance. I generally feel that there is some useful role for anyone on any project, it's all about making it possible to harness their efforts ....
Paul.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 3 years ago.
Improve this question
I work as a PHP developer, and my boss is driving me crazy. He calls me every now and then, gives me tasks even when the previous tasks haven't been completed. Forbids me from meeting with clients and tells me incomplete specification verbally or over the phone, and expects to have working applications from the first deployment. Can any body suggest a way to curb this guys enthusiasm?
document what you are spending your time doing.
document requirements as they come in (through whatever channel)
provide both of these to your boss as frequently as necessary in order facilitate conversation about missing requirements or prioritization of your time
Start looking for another job and give him a pre-notice. This will curb his enthusiasm. Some people are to be avoided at all costs.
//Only two things are infinite, the universe and human stupidity, and I'm not sure about the former." -- Albert Einstein
Talk to him about the communication problems. It's usually a two-way problem. If he's giving you incomplete specs, why aren't you asking for more complete ones?
If you just want to stick it to him, then find a better job. If you're committed to making it work, then put some serious thought into the root of the problems you're having, and get him on board with helping you solve them.
I'd suggest having some sort of task list somewhere so that when your boss adds something to this list, he can see what you are doing and have estimates for how long some things will take.
As for the requirements being incomplete, this is why some like Agile approaches where you would get your boss in to test something regularly to verify that it is acceptable before putting something into production.
As others have mentioned who can type faster than me, one thing you can do is to try setting up some type of project management/bug tracking system for him to use. I've used FogBugz with success which requires no installation and has a free plan for small teams. You can setup an email account that he can send requests to.
When proposing this to your boss, do it in a way that will show that your goal is to help keep track of your tasks and priorities. But also be prepared for additional work. If he is agreeable to it he may want to see frequent status reports and where you are spending all of your time.
If you can get more complete specifications this can help drastically. It'll help you understand the project better. It'll help you estimate time for completion. And my favorite -- it'll help prevent double-work. There's nothing worse than getting incomplete requirements, building something wrong, and then finding out that you need to redo it.
If you can I'd try to find a good way to suggest this to your boss without stepping on any toes. If he understands the benefits he may be more open to the idea of being more thorough with you. What it'll really do is make you more efficient at your job.
Oh, and you may want to take a look at my answer to this question relating to time estimation:
How am I supposed to know how many days something will take?
The other suggestions of keeping things in writing at least for yourself are valid. However, for the kind of boss you describe with which I have some familiarity, it may not work.
Some bosses just never learn or already know what they are doing is wrong and do it anyway because to them, saying "yes" to the client and getting a signoff on a bigger contract today is more important than programming methodology that could delay a payment for 3 weeks (testing? why do we need to test? you tested it, right? two weeks for function X?! all you have to do is add a button, right?).
To many bosses, programming is just one of the tools needed to run their business. To them, it's a business to make money and programming methodology/quality is secondary.
What you do to keep yourself sane and safe if you want to stay with this boss is at least keep written records of every request and when problems come up, remind him why they happened. If possible, ask that all specs be delivered at least by email instead of verbally.
You always have the choice of going somewhere else if things don't improve.
Good luck.
I have struggled with this same problem at a few jobs.
I've observed that there is a type of programmer who can survive in this environment: one who simply works on whatever the boss says is the priority of the moment, and doesn't worry about deadlines at all. Just keep putting things on the back burner, and resume working on them the next time the boss asks for it. Endure the occasional tirade from your boss when something he asked you to start 12 months ago isn't done yet, because you've been working on whatever he gives you.
If you aren't comfortable with that sort of existence (and I would not be), then the best strategy I can suggest is that you have an open conversation with your boss about it:
It's important to me to do a good job,
but I feel like the work requests are
coming in faster than I can complete
them. I know that priorities
inevitably shift, so I'd like some
help from you to understand when this
happens, and how to balance my time
among the tasks to best meet your
needs. Can we talk about how to stay
in sync about priorities?
If the tasks can't be prioritized,
then I need you to understand that I
can't be productive on so many tasks
at once. It's just not humanly
possible. Have you considered hiring
additional programmers so we can
balance the workload and get it all done
in a timely way?
If he tries to stall or contradict or tells you to "work smarter, not harder" then you can escalate:
Let me put it another way: if you continue to give me tasks at the rate you have been, but don't prioritize them clearly, then you are going to hire another programmer. Whether this hire is your second programmer, or else my replacement, is up to you.
Basically, communication is good. Start by being diplomatic and phrasing everything in terms of how you contribute to his success. But increase the degree of bluntness until it sinks in.