Yahoo Pipes Text in/ISBN out

[Gary Green]

Hi. I'm pretty new to Yahoo pipes and I'm trying to work out how I'd set up a pipe where the user types in some text, it connects to a catalogue, does the search and from the output the ISBNs in the results lists are fed into another catalogue to pull in further related info from the 2nd catalogue. Does that make sense? Unfortunately, I can't work out how to pull the ISBNs from the results list and pass them onto the next pipe. I can do the initial entering the text and pulling the results out of the catalogue bit. I looked at the ISBN thread on the forum and don't think this is what I'm after. Any help at all would be appreciated. Thanks.

[Mark Watmough]

Hi Gary,

I think you will be able to submit this via a yahoo pipes URL, but without your specific example to work on, it's difficult to provide a definitive answer.

Using one of the Yahoo! pipes team's examples which searches wikipedia for given words, you can see how you can pass values via a URL:

http://pipes.yahoo.com/pipes/pipe.info?_id=KGSh5N9S3BG6AuwU9YS63A&a...

If you can let me know the id of your current pipe (after you 'publish' it) then I, and probably some other more technically minded 'masher', could give you more help.

Thanks

[Owen Stephens]

Hi Gary. What format do the records come back from the catalogue in? As Mark suggests if you can post a public version of what you've done it be easier to comment

[Gary Green]

Hi Mark & Owen
Thanks for the replies. As I say, it's early days for me with Yahoo pipes, so I'm still getting my head around things. I thought a more sensible/less complicated approach might be to merge the data from 2 catalogues based on the ISBN, (to provide merged records) but I think I'm still onto a loser with that idea. I've published the basic pipe and am going to continue tinkering with it.

http://pipes.yahoo.com/pipes/pipe.info?_id=85c0c3bdf8a32d3f66b57f9f...

Please don't feel obliged to spend a lot of time on this, but any pointers would be welcome.
Thanks.

[Owen Stephens]

Hi Gary,

I like the idea :) I guess I can see a few issues. The biggest problem you've got is that the records you are getting from WorldCat and OpenLibrary via your websearches don't (reliably) contain the ISBN. I wonder if there are any other ways of getting the data out of the sources you are looking at that would include the ISBN (where available) - RSS feeds? OpenSearch format? It doesn't have to be in a field of it's own, just somewhere in the response.

There's some other stuff to think about - do you want to deduplicate just those things with identical ISBNs, or do you want to try to merge at a slightly higher level (e.g. hardback and paperback versions of the same book which would have different ISBNs but are essentially the same thing from the user perspective) - this would make things a bit more complicated, but there are some services like xISBN and ThingISBN that would help acheive this.

The other issue you'd have is that not all books have ISBNs - but I'm not sure how much of an issue this would be for you?

[Gary Green]

Hi Owen

I'm still working on this. I'll have a look at xISBN & ThingISBN and see if they can help. I am hoping to basically create a merged record with the ISBN as a unique number, but so that it brings in useful pieces of information from different sites eg. basic record from site 1, add any missing info to the record from site 2, have a link to the full text from site 3 (eg Google books).

After your comments I thought I might be aiming a bit high, so I decided to do something less complicated on a bit of a different track...

I'm a fan of horror and anything ridiculous, so I came up with this pipe while I was tinkering at home/out of hours. The idea came about from reading 'Pride & Prejudice & Zombies', which just seemed as if the author had stuck in a few 'zombies' , 'unmentionables' and tweaked the text a bit (I think a film of the book might work better than the book itself). I decided to set something up to take the synopsis & title of a book and tweak it into a Jane Austen horror novel.

http://pipes.yahoo.com/pipes/pipe.info?_id=70cd9a811adcb8a0e920a3cd...

I'm quite pleased with the result, which is set up as a default on 'Northanger Abbey'. However, the 'replace text' bits are too specific to the synopsis I've chosen. My next step, is to develop the pipe to be able to search a specific catalogue for any book based on author/title, pull in the synopsis or a portion of text (eg chapters), extract the terms, use some of the extracted terms to feed into the 'replace' module and replace the text with various 'unmentionables', 'lycanthropes', 'hairy beasts' etc. And then display the revised version. I plan to embed the pipe in a page too. I'll let you know how I get on.

It doesn't have any practical use, but I found it fun to do and was a good learning experience.

[Gary Green]

Hi
I've made a bit of progress on my original text in/ISBN out Yahoo pipe.

http://pipes.yahoo.com/pipes/pipe.info?_id=85c0c3bdf8a32d3f66b57f9f...

I've got the stage where I'm able to pull out the ISBN from Worldcat and put it in it's own isbn tag. After using the Regex module (I used this to strip out the ISBN) I need to change the output to text (rather than output as items) to feed it into other catalogues as an ISBN search. I've looked at the Yahoo pipes forum & the common suggestion is to use a loop containing a string builder to output text. However, this doesn't appear to work.

Any thoughts would be welcome.
Thanks.

[Gary Green]

Sorry - correct link is http://bit.ly/171e5W

[Owen Stephens]

I can't see the RegEx in the pip you've linked to?

Anyway - some more general comments:

The pipe I can see at the moment uses a Yahoo search block which is probably not the right format here. You'd be better off with something that fetched more structured data - if this is available (which it may not be).

Tony Hirst has done some demonstration pipes for Open Library which show how you can use the Open Library API to get more structured data out http://ouseful.open.ac.uk/blogarchive/014455.html - but it isn't simple I'm afraid.

Looking at what you are getting at the moment, there looks like some possibility of scraping the ISBN out of some of the Worldcat results where it appears in the link - but Worldcat uses the OCLC number instead here quite often (I'd guess as a preference to be honest)

Worldcat have or will have an API which will return results in RSS - see http://www.worldcat.org/devnet/wiki/BasicAPIDetails - this would make it much easier to do what you are trying to do here.

I realise that this may not be a huge amount of help to you - all I seem to be doing is saying 'try doing it another way, which is more complicated' - sorry :( - but if you can get the structured data out, this is going (overall) to be much simpler!

btw I loved the Jane Austen/Horror mashup :)

[Gary Green]

Hi Owen
Thanks for the reply.

The Regex module appears between the Rename & Loop pipes. (it mentions 'item.isbn')

I'll have a look at the documentation you mention. I'd been trying to work it all out based on a book by Tony Loton, which was a good starting point, but left some important gaps. There's definitely a market for a more detailed Yahoo Pipes book out there. I'd also been using the onsite documentation/discussions, which only really gives more details based on specific problems and not from a "this works this way and this is why..." point of view.

As you say, it looks like achieving what I want to will be more complicated, but it's all part of the learning curve and I do appreciate your help on this. I'll keep at it.

I think developing v2 of the Jane Austen/Horror mashup will be much simpler. Glad you liked it.

[Owen Stephens]

Hi Gary - sorry I'm not seeing any of the modules you mention in the pipe you've linked to - all I see is the text input, two search modules, and a union module (unless I'm missing something obvious)

[Gary Green]

Hi Owen
That's really odd. When I click on the bitly link above it takes me to the correct pipe. Please try the full link, here.

http://pipes.yahoo.com/pipes/pipe.edit?_id=85c0c3bdf8a32d3f66b57f9f...

This pipe is built as follows: Text input; Yahoo search; Rename; Regex; Loop (contains another pipe called 'simple combo'; Pipe output.

Thanks. Gary

[Owen Stephens]

Very odd - this is what I see at that address: