Related
**update: horray! so it is a journey of practice and understanding. ;) now i no longer feel so dumb.*
I have read up many articles on REST, and coded up several rails apps that makes use of RESTful resources. However, I never really felt like I fully understood what it is, and what is the difference between RESTful and not-restful. I also have a hard time explaining to people why/when they should use it.
If there is someone who have found a very clear explanation for REST and circumstances on when/why/where to use it, (and when not to) it would benefit the world if you could put it up, thanks! =)
REST is usually learned like this:
You hear about REST being using HTTP the way it was meant to be used, and from that you shun SOAP Web Services' envelopes, since most of what's needed by many SOAP standards are handled by HTTP in a simple, no-nonsense way. You also quickly learn that you need to use the right method for the right operation.
Later, perhaps years later, you hear that REST is more than that. REST is in fact also the concept of linking between resources. This often takes a while to grasp the full meaning of, but when you learn this, you start introducing hyperlinks into your responses so that clients can navigate your system without being coupled to how the server wants to name its resources (i.e. the URIs).
Even later, you learn that you still haven't understood REST! And this is because you find out that media types are important. You start making media types called application/vnd.example.foo+json and put hyperlinks in them, since that's already your understanding of REST.
Years pass, and you re-read Fielding's thesis for the umpteenth time, to see if there's anything you missed, and it suddenly dawns upon you what really the HATEOAS constraint is: It's about the client not having any notion of how the server's resources are structured, but that it discoveres these relationships at runtime. It also means that the screen in front of the user is driven completely by what is passed over the wire, so in fact, if a server passes an image/jpeg then that's what you're supposed to show to the user, not an error message saying "AtomProcessor can't handle image/jpeg".
I'm just coming to terms with #4 and I'm hoping the ladder isn't much longer! It's taken me seven years.
This article does a good job classifying the differences in several http application styles from WS-* to RESTian purity. What I like about this post is it reminds you that most of what we call REST really is something only partly in line with Roy Fielding's original definition.
InfoQ has a whole section addressing more of the "what is REST" angle as well.
In terms of REST vs. SOAP, this question seems to have a number of good responses, particularly the selected answer.
I would imagine YMMV, but I found it very easy to start understanding the details of REST after I realised how REST essentially was a continuation of the static WWW concepts into the web application design space. I had written (a rather longish) post on the same : Why REST?
Scalability is an obvious benefit of REST (stateless, caching).
But also - and this is probably the main benefit of hypertext - REST is ideal for when you have lots of clients to your service. Following REST and the hypertext constraint drastically reduces the coupling between all those clients and your server, which means you have more freedom when evolving/developing your service over time - you are not tied down by the risk of breaking would-be-coupled clients.
On a practical note, if you're working with rails - then restfulie is a great little framework for tackling hypertext on the client and server. Server side is a rails extension, and client is a DSL for handling state changes. Interesting stuff, check it out here: http://restfulie.caelum.com.br/ - I highly recommend the tutorial/demo vids they have up on vimeo :)
Content-Type: text/x-flamebait
I've been asking the same question lately, and my supposition is that
half the problem with explaining why full-on REST is a good thing when
defining an interface for machine-consumed data is that much of the
time it isn't. OK, you'd need a really good reason to ignore the
commonsense bits (URLs define resources, HTTP verbs define actions,
etc etc) - I'm in no way suggesting we go back to the abomination that
was SOAP. But doing HATEOAS in a way that is both Fielding-approved
(no non-standard media types) and machine-friendly seems to offer
diminishing returns: it's all very well using a standard media type to
describe the valid transitions (if such a media type exists) but where
the application is at all complicated your consumer's agent still
needs to know which are the right transitions to make to achieve the
desired goal (a ticket purchase, or whatever), and it can't do that
unless your consumer (a human) tells it. And if he's required to
build into his program the out-of-band knowledge that the path with
linkrels create_order => add_line => add_payment_info => confirm is
the correct one, and reset_order is not the right path, then I don't
see that it's so much more grievous a sin to make him teach his XML
parser what to do with application/x-vnd.yourname.order.
I mean, obviously yes it's less work all round if there's a suitable
standard format with libraries and whatnot that can be reused, but in
the (probably more common) case that there isn't, your options
according to Fielding-REST are (a) create a standard, or (b) to
augment the client by downloading code to it. If you're merely
looking to get the job done and not to change the world, option (c)
"just make something up" probably looks quite tempting and I for one wouldn't
blame you for taking it.
For example team A and team B are working on different applications that need to implement a similar feature. The feature in question relies on a database and the database is under the control of team B. Even though the user interfaces of the two applications are based on different technologies, the functionality is supposed to be roughly the same. Both teams have their own requirements and design documents. The functionality can be changed based on feedback from either team but then both teams have to update their requirement and design documents.
The teams are geographically distributed and members of each team itself are also geographically distributed. Both teams work with the same client entity but different people. Each team has their own business analyst (requirements specialist).
I am trying to make the technical communication between the teams more formal than email so that we can avoid misunderstandings.
How do you make sure that if team B changes the database and or the feature functionality, the other team gets properly notified about it? Do you use some formal text based documents such as interface contracts? Can you share any templates for those? Or do you use some other mechanism?
A couple of things from my own experience (which sounds very similar to yours)
You should try and have a single design document for the database part of the solution which as djna suggests should be posted on a wiki or similar, with a defined public contract for interaction with the data. This is a good step in the right direction, as it will give everybody a kind of 'shared vision' which helps people converge towards doing the right thing. The contracts should try to ensure that the data access is done in a standardised way.
However, from experience, the code does not always follow the spec exactly, so i would also assign a single owner from one of the teams, whose responsibility is the integration of both systems to the database.
i would then implement a continuous nightly build process with tests, and this build should include the database. This will hopefully flag any issues earlier in the process.
From the project i worked on, you may still have occasional disagreements and breakdowns, eventually we merged both teams. This was the best solution of all for us!!
Hope this helps a little
What about having a Team site (both as one team) or a Wiki so that both teams are aware of the change.
Regular stand-up meetings. Via a conference call. Stand-up == brief, highly focused, information centered. Delegate discussion to individual discussion outside meeting, reporting back at next.
There does need to be an overall authority though, to mediate where agreement cannot be reached and to ensure overall solution integrity.
I agree with Wiki or other collaborative site for publishing the current reality.
I'm trying to figure out how to best implement a public data hosting service.
How do websites that let users upload pictures enforce their terms of service regarding obscene pictures? Do they use image processing algorithms to flag potential violations (too many skin-colored pixels)? I think Imageshack looks at the websites that their pictures are hotlinked on, and checks for keywords. If it detects anything porn related, then it removes the picture and bans the account. Are there other methods?
Is enforcement largely automated or is it based more on user reports?
I suppose it depends on the scale of your "public data hosting service".
If it's something small with maybe a couple hundreds pictures per day flowing in, you can moderate them on your own.
If it's a couple hundred thousands you'll need an amount of human beings sorting the weeds out. It's either a moderator team or users themselves who submit abuse reports.
Which one to go, can be dependent on your budget/financial success of your service as well as on the type of the service. If it's something simple like Rapidshare where one does not see what the other does, the chances that users will see each others content and through this notice and hopefully report unacceptable content are small. If it's something very social like Flickr you can bet on it reports will be flowing in.
I suppose you could automate something but it's almost an impossible task. You can't automatically detect porn. You can't automatically detect images violating copyrights - making footprints of copyrighting material in order to compare them with the uploaded stuff is a real challenge for companies with resources like Rapidshare, Youtube and others. For now this kind of work can effectively be done only by humans.
There are also legal issues to it. In some countries the service owner is not liable for what users contribute (well, if he's cooperative enough to delete certain content at request), in others he will get the charges himself for not having premoderated all the incoming content. Also think of this with regard to whatever and wherever you are going to launch.
I don't have links, but while it's certainly a difficult task prone to errors, software to detect improper content does exist. Or at least that's what the Security Manager at NASA told me - if if was just a means to scare me I don't know ;-)
Closed. This question is off-topic. It is not currently accepting answers.
Want to improve this question? Update the question so it's on-topic for Stack Overflow.
Closed 11 years ago.
Improve this question
We need to involve our customer development partners in our development process. We're more or less following Agile methodologies. Some customer partners are remote, others closer. We need to minimize travel costs.
Our customers are in health care and tend to be busy, expensive, and hard to schedule.
What practices and technologies have worked to support customer involvement? We're using phone calls, phone conferences and email. We're curious about leveraging wiki techniques and would love to hear what's worked for others.
it doesn't matter whether the customer is in the same cubicle or halfway around the planet, except for communication delays - the critical factor is availability.
a customer that is too busy to answer your emails for several days is going to cause your iteration to be late, or fail
the customer has two critical commitments for agile:
available to answer questions in a timely manner
not to change their mind/priorities during an iteration
the customer must commit to a reasonable service-level agreement (SLA) on availability, e.g. 1-hour response time, or 24-hour response time, etc., and you will need to adjust all estimates and schedules by the lag factor. If the customer will not commit or does not follow through, cancel the iteration and re-plan, bringing the customer's commitment to the forefront again. Do not just "guess" at what you think the customer might want.
Bottom line: without a customer commitment, agile will not work.
My experience with Agile methods is mostly for desktop applications. When our customers are remote, we've spent time to get an engineer to the customer site to configure/install a demo rig. The engineer works with the customer on a test and demo setup/plan that will provide an environment that the customer believes replicates the important aspects of the deployment environment but isolates the demo system from existing infrastructure (so that we can push updates whenever we need to). The engineer also sets up deployment systems to move our applications into production, so that we can "deploy" without being on site. Our applications can self-update (either for each release or each build) and we carefully instrument the releases to log all errors and submit all crashes as bugs to our bug tracker. This way we at least know what went wrong, even if we don't know what's going right.
For each release/build that shows up on the customer's test rig, we provide a (short) screencast, narrated by the project lead or primary developer, demo-ing any new features. The release notes contain any long-term issues or questions we want the customer to think about (i.e. issues that can't be resolved immediately by a phone call or email), and the application displays these notes for the user.
Finally, and possibly most importantly, we get the customer and/or the customer's liaison an account on our calendar server and configure their calendar app to make use of that account. This then goes both ways--we can schedule time (on site, phone, email, etc.) with the customer and they can do the same with our developers.
One option: Install a customer proxy at the "customer partner" site who can extract the information that you need when those customers are available. Have these proxies build the solid relationships that allow them to represent the customer view. Their time is all yours. And when questions arise that they cannot answer, they have ready access to your customer partners - even if in the coffee line.
The whole point of the customer in agile is to have open and free discourse with the developers (IE immediate feedback). If your actual customers cannot provide this, then you need an intermediary/proxy that can fill this role. You don't need actual customers, you just need someone that can represent the customers' interests well enough to meet your customers' needs.
Just a few ideas:
If you do choose to use a Wiki, make sure it supports a whole-wiki-wide "recent changes" list, and preferably one that is specific to the users. The less distant from development people are, the more likely to have email as a metaphor for their computer use. If they can't immediately tell when there's something new for them to see, they will never explore it. You also preferably need ways to signal to them that you need their attention to matters, or they will treat changes like CCs.
I'm a big believer in creating video screen captures of interactions (narrated) and distributing them to users. Unlike a real demo, customers don't feel like they need to interrupt, and they can rewind and re-watch the same interaction over and over, paying attention to little details.
Finally, if you do distribute prototypes, make sure to send someone (or at least a screen sharing session) to see how the prototypes are used. Contextual design is effective. You can count on people using your prototype differently from the way you expect, and you have to understand how they use it to really understand where the issues are, even if they don't report them.
Have you considered something like LogMeIn.
This would allow customers to either log-in to a PC on your network already running your application, or alternatively allow you to install/update the application on one of their computers.
This would solve the remote customer issue and would also support the ongoing continual customer feedback requirement in the agile process.
I used it a previous company for technical support, but there is no reason (except maybe cost) that it would not work for your situation.
It is also a great way to actually see how users are using your application and therefore find out what works and what doesn't.
First of all, make sure that you have a product manager or a product owner close the the developers. This person will be managing the relationship with the customer.
Then, the product manager can demonstrate the product to the customer at the end of each iteration and also ask customers question when the developer need feedback to implement a user story.
It is amazing the positive feedback you can get from customers when you involve them.
We did not use a wiki and most of the communication is done via E-Mail, phone, and a screen sharing application (we are using GoToMeeting, but there are tons of alternative out there).
You should probably do a kick-off once with everyone at one place. Face-to-face time is invaluable. That includes all developers. Prepare some metaplan questions, but also have enough time to just mingle.
I think by most definitions of Agile processes that have high dependence on customer involvement you've already missed "best practice", which would be for an on-site, and preferably "in-team" customer present at all times. So I suppose we're looking for a "next-best practice". :)
There's the possibility of introducing a "proxy customer" on-site. I have to admit to being very sceptical about the value of a proxy customer. I'm concerned about the risk of introducing some sort of second-rate and otherwise unnecessary business analyst function to the mix, with the increased signal-to-noise ratio and potential for garbled messages. It also carries the risk of allowing busy real customers to reduce their involvement in the process, which is likely to lead to dissatisfaction. I wonder if there might be someone with good domain knowledge who has recently retired and might be available to act in this capacity as a consultant?
Communication bandwidth with remote customers is astonishingly lower than face-to-face, something I had not fully realised until I started dealing with users in another country. Even with video the loss is significant.
How long are your iterations? How hard is planning iterations? Might it be easier to go for longer iterations and get more planning done less frequently, or reduce iteration length and go to smaller, but more frequent planning sessions? Are more than one customer involv
Do you have a useable and available build at the end of each iteration? Is there time for involved users to have hands-on time before the next planning session? Keeping users engaged by shipping frequently would seem on the surface to be a Good Idea, which perhaps legislates for small frequent iterations (a week? two weeks?)
The wiki idea might work: have you looked at the FIT Framework? It's a sort of integrated acceptance test/wiki, which might help in getting acceptance tests from remote customers. I think I'd also look to provide some sort of (separate or integrated) "project dashboard", possibly pushed regularly to key customers as well as available on demand. use it as a substitute for things like post-its on whiteboards, Big Visible Charts and the like. There are a number of open-source or low-cost options that may serve - writing your own simple alternative need not be too time-consuming or costly, either.
Above all, remember that "Agile" is a kind-of catch-all label for developments that are carried out with an emphasis on the values and principles espoused in the Agile manifesto. What is considered "best" in one situation may not be so in another. If you understand the principles and regularly review your methods with a critical eye then you're probably going to be close enough to the best practice application to your situation.
I haven't looked at it for some time but with Beck and Fowler on the author list, there should be something useful in Planning Extreme Programming.
In my previous position #drchrono.com I aggregated data/feedback/iteration requests from 20,000 clinicians across the country. The best way to do this is to to evangelize a site like uservoice.com. I held "daily live web demonstrations" with sometimes 50 to 100 doctors (doctors signed up right from our website). In these demos I would demonstrate our current product and evangelize user voice to drive their feedback into a useful tool for our development team. All of this was done remotely and led to a 1,400% overall increase in recurring revenue growth.
Why is this archaic format still used in the face of easier-to-use technologies? Does it provide some benefit that I'm not seeing? It seems that a large amount of vendors still provide data only in this format, instead of something more manageable and easier to use such as XML; at the least it would make sense to me to offer both formats.
Also, what are some good ways to deal with and utilize EDI when you have no other choice but to use it? Something like BizTalk is out of the question as it's far too expensive. Are there any free/open source applications that make EDI easier to work with?
EDI is not that hard to understand once you familiarize yourself with the delimiters it uses. You might ask yourself as well why anyone would still be using CSV or tab-delimited data.
The answer is probably that those formats are "domain specific languages" defined by committee and standardized in a certain industry, and that a lot of money has already been invested in supporting those formats. Where's the business case to throw that all out again?
One word, Inertia. Developing the EDI formats by committee between various companys and organisations with different agendas was a nightmare (sad to say I have been there).
Asking them to abandon these with yet another round of committees agreeing web service API standards is going to take even longer, how do you sell the idea of replacing one electronic format with another to a non-technical board? What possible busness advantage does it give them. Originally the benefits of electronic exchange were clear but replace one with another is not. We're talking really big companies here.
You may be interested in the following project:
http://bots.sourceforge.net/en/index.shtml
Google code archive
A little information for all interested. EDI is basically a design by committee data exchange format that not only set out rules for data formatting (like XML), but also set out to define each document that could possibly ever be sent between 2 companies. So for any piece of data that could be exchanged between companies they came up with an exact definition of what was supposed to be in each of these documents. Of course, nobody could foresee every piece of data that 2 companies would want to exchange. So you end up with companies using fields that were defined for 1 thing, being used for some other piece of information.
What you ended up with, is an extremely convoluted data format, in which many people using it don't follow the standards, because they need to send custom information, which the standard doesn't account for. So in the end, you still need to talk to each company you want to deal with, and find out all the little idiosyncrasies of their implementation, just as you would have to do if you went to someone with a custom XML interface. Except that in the case of EDI, the format is hard to parse and even harder to write well, so you end up doing a whole bunch of work just to send a document, when doing the same kind of think with having a custom XML solution would have resulted in many times less problems.
And switching to XML would give you what - a slightly easier to debug line format?
Generally you set it up and leave it, there isn't a lot of need to play with the raw EDI feed, certainly not enough to abandon the standard and start again.
There are lots of standards, like FAX that could be made more readable but no real pressing need to change them.
Because it's a formally established Standard (in fact a very large and comprehensive set of standards). And that's one of the claimed benefits of a standard - you won't need to change anything for a long time.
And to change it, it takes agreement between two or more (often thousands and thousands more) trading partners (including maybe all of your competitors) to agree.
EDI formats have much higher signal-to-noise ratios (because they were designed back when that was considered important.) Someone who knows and understands EDI will look at your XML and say "Where's the beef (data)?"
Very few developers write their own parsers. There are many good mappers available (and many legacy and enterprise apps come with them built in). So there's lots of relief available for your pain (including at least one Open Source app on SourceForge).
"If it ain't broke, don't fix it."
Most of these organisations are processing vast amounts of data using EDI, and aren't about to change to something more modern without a compelling reason. And making things easy for third-party developers doesn't usually qualify, sad to say.
IMHO there are several problems with EDIFACT.
It is not easy to parse or generate an Object model from it. This is probably not a big problem anymore as there are now good system around that do it for you e.g. smooks.org
It is not easy to read. You get used to but XML is a lot easier to read
Validation isn't that easy (compare that to validating XML)
There are far too many different versions and flavours, D95B, D96B, D00A, D00B etc.
But I think the biggest problem is that everyone is using the standards differently. They use the same 'format' but the fields are defined differently. We use EDIFACT to send and receive messages from Container Terminals and they all have slight differences. They would e.g. all use a D95B CODECO but for some terminals a certain segment is mandatory while for another it is optional or even not allowed to be there. Then you have segments that are used the same but the content in it is different.
So to summarise it: It is a pain in the neck.
EDI is a very compact format and is often used to keep bandwidth usage in data exchanges as small as possible. The German customs offices for example use it in their ATLAS system to exchange a very high volume of data every day.
It is hard to parse and hard to read, but if the size of the resulting data matters, it can be a good choice and is supported by most of the bigger business applications.
Legacy Support
EDI is prolific in many industries. It would be prohibitively expensive to replace an already-working technology with a newer one.
Consider this, Walmart uses EDI to communicate with its vendors, stores, distribution chain, etc. I'm guessing they deal with tenss of thousands of vendors. Every one of them has sunk thousands of dollars into EDI technology. If Walmart decided to switch over to XML, its a decision that affects thousands of companies, not just Walmart.
This is true for any EDI user. After all, it's a standard used between trading partners.
I agree, EDI is a pain to work with. But 'back in the day', that's all we had.
Edifact is one of the best standards when it comes down to document interchange.
Most problems come from tradingpartners sending non standardized documents.
Yes it's a bit odd format and is tedious to work with if you don't know the ins and outs but that goes for XML as well.
You really want XML over Edifact? Look at the bloated, hard to read XML standards peppol (pan-european public procurement online) is working on.
Yes it's working nice and dandy if you don't have any errors in the systems, troubleshooting edifacts is so much easier once you get used to the format than troubleshooting UBL documents.
You say you have $0.00 to use on the project?
You really should look into the amount of manual work done in your company and the costsavings EDI can offer some cost benefit analysis can be mighty handy.
What types of information can be exchanged via EDI?
A variety of types of business information exchange is available via EDI including:
-•Booking information
-•Bill of Lading information
-•Invoicing
-•Electronic Funds Transfer
-•Arrival Notice Information
-•Shipment Status Information
How would choosing EDI benefit my company?
-•It streamlines the communications process between you and APL
-•It eliminates the need to rekey data, thus eliminating errors and the need to recheck information
-•It eliminates paper handling and the need for document storage
-•It improves the turntime and the accuracy of your data
-•It eliminates the need for faxing
One solution, although it will cost you, is to go to a company like ADX, which has tools you can use to convert EDI formats to more pleasing formats like CSV. Depending on the volume and type of transactions you are doing, this can be both affordable and a lot less stressful. I've used their products in the past, and while they are a bit of work to set up, they do work quite well, and are very stable. Because of the history of EDI, you could probably find hundreds of other companies that offer similar services.
EDI has been around since before XML. Apart from the fact that two parties can pre-negotiate the EDI format that works for them both you must also consider the part of the VAN (value added network.)
In some cases the VAN performs validation of the message, or even reads the message and performs actions on it, such as copying it to additional parties based on its content.
The only reason really to use EDI is because "that's the way it's always been done", and therefore there is a lot of existing infrastructure around to support it. Why switch to XML when there is no need? And how is to say XML wont be replaced by JSON which will then be replaced by something else?
Another reason is that being business messages such as order. invoices, credit notes etc there is a lot of financial worth in the transactions and they need to be secure but perhaps more importantly they need have end to end validation and verification as well as non repudiation.
For example i send you an order for 1/2 million Euros worth of goods, you send me the goods, then i "lose" the order information and tell you i am not paying. The combination of the standards and the VANS make this almost impossible or at least with so much of an audit trail that it the problems could be tracked. This is why the "Oh let use xml and the internet instead of EDIFACT and the VANS" tend to fail. As someone els answered, Inertia, but it is an inertia founded in a stable effective, secure, reliable and well understood system.
Doing it on the cheap is not always an option.
If it is any consolation when i first implemented EDI in '87 there was virtually no software around and so i got the Interbridge tables and wrote my own parser for the UK TRADACOMS standard using Cognos software on and HP Mini, and it worked fine. Assuming you are trading with other EDI partners the cost probably comes at the point of needing to use a VAN.
I've used EDI (ANSI X12 and EDIFACT) in 2 projects about Maritime Transport Logistics and found them to be very useful since most Ocean Carriers and Trading Partners accept them as the standard way of communication between their different systems.
So EDI format is still used and will continue to be used since it's a stablished standard and thousand companies have developed systems around them, and replacing them is a really big deal.
I've had to use EDI as well and I agree. We used BizTalk to map it which worked well. Many system are built on EDI(well before XML).