anybody have used Jitterbit as an ETL tool? any reviews/tutorials?
I have used Jitterbit and I have found it easy to use and they are a pretty responsive company to work with.
Since the question was posted they have added several webinars to their website that give you a good introduction to what you can do with the product:
Jitterbit Integration Webinars
Jitterbit Developers added some small Tutorials:
link text
It is pretty good and easy to use.
Very good in transformation. It has auto mapping tool, these is a way you can map source and destination pretty easily.
GUI is pretty strait forward and very user friendly.
It has lot many file operations. You can access FTP, Share, Local file, HTTP, LDAP etc etc
Web service is also very well written.
I have used Jitterbit for integrating local PostgreSQL database with the salesforce
Related
I'm quite new to the field of computer science but I think I've got a pretty decent idea for a website to aid classroom CS learning and collaboration. I'd really like to develop the website from the ground up and make it a sort of pet project in hopes of eventually getting it out on the web for free. Hopefully I can get some teachers to adopt it for use with their classes.
The problem is that I honestly don't know where to start. I've got the idea but I don't have enough formal education to guide the implementation of my idea. The site should have quite a bit of functionality in the long run. I'll need to be able to store user and class data/files as well as offer discussion boards and other things.
Without getting into too many details, what is the best way for me to get started? What languages and databases should I be most interested in as I build the site and ensure scalability and future functionality developments? I would really appreciate any information you could give me on how to structure the project/stack as I don't have much of a clue at this point. I have the idea. Now I just need a little bit of help getting started.
Thanks!
There are definitely already projects out there that will (more than likely) do everything you're currently considering. That said, there's immense benefit in doing a project like this for personal development - you get to learn, and you expand your public portfolio. If you run the project as open source, you can also demonstrate your ability to work with others. All very good (hireable) attributes.
Are there any programming languages you already know? Are there any that your course is going to be teaching that you know ahead of time?
There are so many different languages and frameworks available to choose from, but I'll only mention a few.
Language: Framework
.NET: ASP.NET MVC
python: django
ruby: ruby-on-rails
I'm a huge fan of django. Python is quite a nice language to learn. I'd recommend django purely from a biased point of view. Python runs on Windows, Linux, and Mac, though you probably don't want to host python on windows (culture more than ability).
Conversely, if you really like Windows, ASP.NET MVC makes building out websites very very easy. Mono does allow you to run .NET on linux and mac, but you might find support lacking, and I wouldn't suggest using Mono for your first project.
PHP is (was?) another popular language for building websites in. There are tonnes of web frameworks available for PHP. Popular opinion seems to be that PHP makes it easier for developers to write bad code, though it is possible to write good code with PHP.
Unfortunately, without knowing a rough direction in which you're headed, it's nearly impossible to offer some concrete advice. Database choice will generally come down to what language and platform (linux/.net) you're targeting. Web server also fits this profile. Once you decide on a language, narrowing down the other choices become a lot easier.
Learn HTML to start with and keep improving as per needed with css , javascript. You won't need more then this.
I have a fairly complex windows service (written in .net 4) with several sub systems that run in parallel.
I have implemented pretty good logging throughout, but I'm feeling I'm needing more info about what each subsystem is currently doing. This would be very useful for times that I need to stop the service for upgrade/bug fixes.
It would be nice to have a gui app that will show me the status for each part of the application that I'm interested in. I've had some ideas for how I'm going to do this, but I'd like to hear some others' ideas as well.
I'm interested in a solution that would be easy to plop down in a future windows service and I'm not looking for anything very complex.
Are there any tools for this sort of thing?
Have you done this yourself?
What about interprocess communication?
Since Windows services can no longer interact with the user session, you'll need to have a separate application that does the interacting for you. Based on the details of your question, I think you understand this.
The big question is how to facilitate the communication between your Windows service and the application. There are all kinds of approaches - shared memory, socket, pipe, remoting, etc. What I have used successfully is WCF. If your UI is going to reside on the same machine as the service, use the NetNamedPipeBinding. If you ever need access from a remote machine, you can change to the NetTcpBinding. I've found this flow chart helpful in binding selection.
.
If you're looking for a more formal framework approach that just straight WCF, have a look at Juval Lowy's Publish-Subscribe WCF Framework, which is described in pretty good detail in this MSDN article. The code is available to look at via the article, or you can download the source and example from Lowy's website here. Go to the Downloads section, filter by the Discovery category, and you'll see it there.
A recent announcement by Google about the Google Prediction API sounded very interesting. It could be useful for a project that is coming up, and would probably do a better job than some custom code I was considering.
However, there is some vendor lock-in. Google retain the trained model, and could later choose to overcharge me for it. It occurred to me that there are probably open-source equivalents, if I was willing to host the training myself (I am) and live without their ability to throw hardware at the problem at a moment's notice.
Last time I looked at 3rd Party computer training code was many years ago, and there were a lot of details that needed to be carefully considered and customised for your project. Google appear to have hidden those decisions, and take care of them for you. To me, this is still indistinguishable from magic, but I would like to hear whether others can do the same.
So my question is:
What alternatives to Google Prediction API exist which:
categorise data with supervised machine learning,
can be easily configured (or don't need configuration) for different kinds and scales of data-sets?
are open-source and self-hosted (or at the very least, provide you with a royalty free use of your model, without a dependence on a third party)
Maybe Apache Mahout?
PredictionIO is an open source machine learning server for software developers to create predictive features, such as personalization, recommendation and content discovery.
Have been looking recently at tools like google prediction API, one of the first ones I got put on to was Weka machine learning tool which could be worth checking out for anyone looking.
I'm not sure if it's relevant, but directededge seams to be doing exactly that :)
There is good free for use service Yandex Predictor with 100000/day request quota. It works for text only, supports several languages and spell correction.
You need to get free API Key, then you can use simple RESTful API. Api support JSON, XML and JSONP as output.
Unfortunately I cannot find documentation in English. You can use Google Translate.
I can translate docs if there is some demand.
How can i write a cloud-aware application? e.g. an application that takes benefit of being deployed on cloud. Is it same as an application that runs or a vps/dedicated server? if not then what are the differences? are there any design changes? What are the procedures that i need to take if i am to migrate an application to cloud-aware?
Also i am about to implement a web application idea which would need features like security, performance, caching, and more importantly free. I have been comparing some frameworks and found that django has least RAM/CPU usage and works great in prefork+threaded mode, but i have also read that django based sites stop to respond with huge load of connections. Other frameworks that i have seen/know are Zend, CakePHP, Lithium/Cake3, CodeIgnitor, Symfony, Ruby on Rails....
So i would leave this to your opinion as well, suggest me a good free framework based on my needs.
Finally thanks for reading the essay ;)
I feel a matrix moment coming on... "what is the cloud? The cloud is all around us, a prison for your program..." (what? the FAQ said bring your sense of humour...)
Ok so seriously, what is the cloud? It depends on the implementation but usual features include scalable computing resource and a charge per cpu-hour, storage area etc. So yes, it is a bit like developing on your VPS/a normal server.
As I understand it, Google App Engine allows you to consume as much as you want. The back-end resource management is done by Google and billed to you and you pay for what you use. I believe there's even a free threshold.
Amazon EC2 exposes an API that actually allows you to add virtual machine instances (someone correct me please if I'm wrong) having pre-configured them, deploy another instance of your web app, talk between private IP ranges if you wish (slicehost definitely allow this). As such, EC2 can allow you to act like a giant load balancer on the front-end passing work off to a whole number of VMs on the back end, or expose all that publicly, take your pick. I'm not sure on the exact detail because I didn't build the system but that's how I understand it.
I have a feeling (but I know least about Azure) that on Azure, resource management is done automatically, for you, by Microsoft, based on what your app uses.
So, in summary, the cloud is different things depending on which particular cloud you choose. EC2 seems to expose an API for managing resource, GAE and Azure appear to be environments which grow and shrink in the background based on your use.
Note: I am aware there are certain constraints developing in GAE, particularly with Java. In a minute, I'll edit in another thread where someone made an excellent comment on one of my posts to this effect.
Edit as promised, see this thread: Cloud Agnostic Architecture?
As for a choice of framework, it really doesn't matter as far as I'm concerned. If you are planning on deploying to one of these platforms you might want to check framework/language availability. I personally have just started Django and love it, having learnt python a while ago, so, in my totally unbiased opinion, use Django. Other developers will probably recommend other things, based on their preferences. What do you know? What are you most comfortable with? What do you like the most? I'd go with that. I chose Django purely because I'm not such a big fan of PHP, I like Python and I was comfortable with the framework when I initially played around with it.
Edit: So how do you write cloud-aware code? You design your software in such a way it fits on one of these architectures. Again, see the cloud-agnostic thread for some really good discussion on ways of doing this. For example, you might talk to some services on GAE which scale. That they are on GAE (example) doesn't really matter, you use loose coupling ideas. In essence, this is just a step up from the web service idea.
Also, another feature of the cloud I forgot to mention is the idea of CDN's being provided for you - some cloud implementations might move your data around the globe to make it more efficient to serve, or just because that's where they've got space. If that's an issue, don't use the cloud.
I cannot answer your question - I'm not experienced in such projects - but I can tell you one thing... both CakePHP and CodeIgniter are designed for PHP4 - in other words: for really old technology. And it seems nothing is going to change in their case. Symfony (especially 2.0 version which is still in heavy beta) is worth considering, but as I said on the very beginning - I can not support this with my own experience.
For designing applications for deployment for the cloud, the main thing to consider if recoverability. If your server is terminated, you may lose all of your data. If you're deploying on Amazon, I'd recommend putting all data that you need persisted onto an Elastic Block Storage (EBS) device. This would be data like user generated content/files, the database files and logs. I also use the EBS snapshot on a 5 day rotation so that's backed up itself. That said, I've had a cloud server up on AWS for over a year without any issues.
As for frameworks, I'm giving Grails a try at the minute and I'm quite enjoying it. Built to be syntactically similar to Rails but runs on the JVM. It means you can take advantage of all the Java goodness, like threading, concurrency and all the great libraries out there to build your web application.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 9 years ago.
Improve this question
I have BS in Computer Science, and thus have experience developing software that runs at the command line or with a basic GUI. However, I have no experience making real, functional, websites. It has become apparent to me that I need to expand my skills to encompass web development. I have been using Ruby to develop applications, but I am aware that it is quite popular for web development. I want to use my skills as a programmer to assist me in developing a personal website for a band.
I have experience with HTML, but very little with CSS. I want to leverage my skills with programming languages to create a website containing pictures, audio clips, a dynamic calendar, a scheduling request tool, and other features common to band websites.
What kind of resources are available for a competent desktop programmer to learn the entire process for developing a website? Is it best to use free CSS templates and WordPress as a foundation for my site or make it from scratch? Should I use GUI tools or write it all in Vim/Emacs? Is Ruby on Rails sufficient for my personal website, or should I consider a more mature development platform?
My main goal for this project is to come up to speed on current web design technology, and actually understand the entire process for building a website.
I think one really important thing to understand in web development is HTTP. HTML and CSS are important, but I think it's more critical to understand the stateless nature of the web, and how each of the HTTP verbs work, and what they can/can't do.
http://www.freeprogrammingresources.com/http.html
A good tool for seeing how HTTP works is Fiddler.
If it's as much a learning exercise as anything then take an iterative approach. Build revise. Build revise. My (very) rough guideline below:
Client
Start with the structure of a website and concentrate on the client.
Use notepad and build a bunch of static pages for your band. i.e. Hand code initially. Try to build all your pages with CSS. No table markup. Then play around with some Javascript to bring things to life. (Navigational menu\ Calendar selections\etc). Learn about how to import and link to Javascript and CSS files.... and how these files are treated re:caching etc.
Try to learn up to the limits of what you can do on the client (generally). Factor in the nuances of 3-4 browsers (Firefox/IE6/IE8/Chrome) re:DOM and client side eventing.
Server
Then start looking for what you might want to change across pages/sessions. i.e. what needs to be manipulated server side. And pick a server side technology.
Start with basic post-back processing. Forget databases at this point. Learn how your framework of choice maintains state..... not just the name of the technology but the real nuts and bolts of it. One of your single greatest assets as a web developer is understanding the state model(s) of the technology you're using.
Then go for a deep dive on the web server technology of choice (and in general). Understand the full request pipeline from client to server and back. This will teach you forms, http and its verbs, web server, filters and modules, server to framework hand off, page and control life cycles, back to the client.
Now start working on dynamic content injection and the like. How to make and use reusable components in your web pages.
Databases, caching, performance and diagnostics.
Then get into into all the fun stuff like ajax, etc. Replace your javascript with jQuery, etc.
Then you got the whole Webservices\XML\JSON\etc side of things to discover.
Resources
Well the web obviously. For client side stuff, going to the sites of companies who make third party web controls can be quite interesting. Asking how the hell they did that? Viewsource is your friend. Look at how they structure and build their pages. Pick a couple of good web designer sites, and you find a plethora of rants about browser wars etc that will give you good (under the hood) info.
Once you hit server side, I'd go for white paper type learning from your vendor of choice for your technologies.i.e. webservers/frameworks/etc. Again find a 3rd party howto/evangelist site (I used to use a lot of "4 guys from Rolla" for example) that will demonstrate how to do various things. Language learning is ongoing. Basically just do the best you can till you find a better way.... and always be on the lookout for a better way.
You really need to understand html, forms and css to get anywhere. I say forms as this will give you the round trip needed to understanding the stateless nature of web dev.
To further labour the point, I have interviewed many people who think you can only have one form on a page and can only have one submit button per form. This is all down to a lack of foundation knowledge.
So for that I'd recommend starting with htmldog.com.
After that, a lot of web development is done with frameworks. Gone are the days where you do it yourself (well mostly) but my above point still stands. You do need to be able to peep under the hood with some confidence.
I've been doing web development for 12 years and started out with Perl on Solaris and Linux. Since then I've also done Java and more recently ASP.NET. However, I'm slowly falling for Django in my private projects.
What I've found over the years is that the inherent problems - cookies, javascript, presentation, state, authentication are all the same but just handled differently. So ultimately its down to you and your language preference. Plus a little of enlightened self interest when it comes to potential employment.
Programming aside, you should also become familiar with web servers (Apache and IIS spring to mind), Http codes and headers, Mime types and encoding and FTP. As well as Javascript (mentioned already), plugins, browser platforms and good development practises such as using Firebug, Fiddler and so on. It also wouldn't hurt to have a good idea of the image formats available, image optimisation, CSS sprites, content compression, caching and the like.
All depends on where you want to start!
For a newbie, I'd pick Django and (obviously) Python. Good, clean language with cheap startup options, low cost IDEs (ie free) and hosting your sites is very affordable.
But that's just a subjective opinion.
If your goal is to
My main goal for this project is to
come up to speed on current web design
technology, and actually understand
the entire process for building a
website.
Then start from scratch in Ruby, PHP, Java, ASP.NET, etc...
When you run into a design problem or just want to know how others have approached something, then look at the frameworks.
Once your up to speed, and your website is starting to grow, then segway into a framework, to get up to speed on the frameworks.
I agree with John on this one.
As you know from your own experience in pursuing your BSc, understanding the basics of any language is what makes you even more capable in expanding that knowledge or specializing.
With that in mind, it would be best to understand the basics of HTML and CSS.
Understanding the syntax and overall language will help in the future when you want to pursue large projects using frameworks like Django and Rails. The basics will also especially help with tweaking CMS' like Wordpress to be more customizable to your needs.
One thing in particular that I'd like to mention is that web programming, like many other forms of programming has its own special structure and "proper" way of doing things.http://www.w3.org is a great way to ensure that your work is passing general web design standards, most sites don't follow this because it is tedious, but from a learning perspective it ensures that you get a nice strong foundation.
www.w3schools.com is also a great resource for detailed help on web programming. Lastly, I like colourful code, so I like using basic text editors such as notepad++ or notepad2 or gedit to do my web coding. GUIs like dreamweaver may tend to fill up your code with extra junk and spaces, so I don't recommend them, but they are still great tools.
Don't bother with Rails yet -- write CGI scripts in Ruby. It will be very similar to what you have done for class.
After you have about thirty of those under your belt, you'll know what you want out of a web framework.
I'm a Computer Scientist and a web programmer and I would suggest you learn both HTTP and CGI:
CGI Made Really Easy
HTTP Made Really Easy
As the titles of the above tutorials claim, they made the concepts "really easy" for me.
Once you've got CGI and HTTP down pat, I'd suggest checking out following sites that provide a wealth of articles and references for web programming:
webmonkey
w3schools
Mozilla Developer Center
Assuming you want to concentrate on writing web apps, then Perl, PHP, Python and Ruby are all fine choices (I myself use Perl predominantly) and I'd suggest doing some research into the popular web frameworks available for each language.
Most importantly, pick something simple as your first web app, e.g. a form and a page that shows the results of submitting that form. Some good examples (using Perl's CGI module) can be found here:
CGI.pm - a Perl5 CGI Library -- see the first set of links on this page.
When you want to start writing web apps that use a database, read up on SQL and popular libraries/modules in your chosen language for database manipulation, especially ORM (Object-Relational-Mapping) interfaces that allow you to deal with records in an Object-Oriented fashion.
Good luck with it! Being a web programmer is fun because your audience is teh intarwebz! :)
If you are starting from scratch as per John MacIntyre's suggestion, you may lean towards PHP. With all of its shortcomings, it does have one really good user manual. It is also easy to get started with and is installed on pretty much every host and goes well with Apache.
Also, w3 schools is good to begin learning about CSS and XHTML but don't forget to check out the specs at W3C.
Also, please read this Stack Overflow question & answers.
For what you're describing, Rails or Django might be slight overkill but it wouldn't hurt to learn them. Django, in particular, might be good because of the notion of a project containing multiple apps (e.g. calendar).
Whether you use a framework or write everything yourself, though, you'll need to know HTML and CSS. CSS is extremely simple if you have a BS in CS...you could read a tutorial and know it in five minutes.