WSS - WebService Specification Schema and LL libraries

Yusuke Kawasaki【川﨑有亮】
http://www.kawa.net/
YAPC::NA 2008 in Chicago
Tuesday, 17 June 2008

Keyboard operation

The slides use JavaScript-based "S6" presentation tool created by amachang.

§1
Introduction

(Q) Do you know how many APIs exist in the world?

programmableweb.com said

780 APIs! (June 17, 2008)
Roughly 0.8 new API everyday

(Q) Do you know how many APIs Google provides?

Counting Google APIs

1. access http://code.google.com/more/
2. enter following code to browser's location bar

javascript:{alert(CODESITE_productDictionary.products.keys.length)}

demo:
source: http://wordpress.chanezon.com/?p=60

Google provides 60 APIs

android gdata blogger feedburner gmail accounts adsense adsenseforaudio adwords ajax ajaxfeeds ajaxlanguage ajaxsearch analytics appengine apps base books calendar chart checkout csearch codesearch customsearch contacts coupons gdgadgets desktop documents earth friendconnect gadgets gears health kml mapplets maps flashmaps gme newsfeeds notebook opensocial orkut safebrowsing enterprise searchhistoryfeeds sitemaps sketchup socialgraph spreadsheets staticmaps talk themes toolbar transitfeed visualization webtoolkit picasa picasaweb youtube

June 17, 2008

One-click translation

  • Arabic
  • Bulgarian
  • Simplified Chinese
  • Traditional Chinese
  • Croatian
  • Czech
  • Danish
  • Dutch
  • English
  • Finnish
  • French
  • German
  • Greek
  • Hindi
  • Italian
  • Japanese
  • Korean
  • Norwegian
  • Polish
  • Portuguese
  • Romanian
  • Russian
  • Spanish
  • Swedish
  • Demo: Good morning!

    The slides are translatable for 24 languages with Google AJAX Language API

    This translation could be better than none.

    TODAY'S AGENDA

    §2
    Who's Kawasaki?

    Yusuke Kawasaki

    川﨑 有亮

    I came from Japan

    日本から来ました。

    Recruit Co., Ltd.

    Gold sponsor of YAPC::Asia 2008 Tokyo
    http://recruit.co.jp/

    Media Technology Labs

    Recruit's institute for research and development.
    http://mtl.recruit.co.jp/

    川﨑有亮 loves

    More detail: http://www.kawa.net/

    CPAN Author

    XML::TreePP

    use XML::TreePP;
    my $tpp = XML::TreePP->new();
    my $url = "http://use.perl.org/index.rss";
    my $tree = $tpp->parsehttp( GET => $url );
    print $tree->{"rdf:RDF"}->{channel}->{title}, "\n";
    print $tree->{"rdf:RDF"}->{channel}->{link}, "\n";
    

    XML and Perl object mapping. (hash/array/scalar)
    Pure Perl, fast and less dependencies to handle XML.
    http://www.kawa.net/works/perl/treepp/treepp-e.html
    Since 2006.02.20

    XML::FeedPP

    my $source = 'http://use.perl.org/index.rss';
    my $feed = XML::FeedPP->new( $source );
    print "Title: ", $feed->title(), "\n";
    print "Date: ", $feed->pubDate(), "\n";
    foreach my $item ( $feed->get_item() ) {
        print "URL: ", $item->link(), "\n";
        print "Title: ", $item->title(), "\n";
    }
    

    Pure Perl and easy to handle feeds without heavy dependencies.
    http://www.kawa.net/works/perl/feedpp/feedpp-e.html
    Since 2006.02.24

    XML::FeedPP, cont.

    my $feed = XML::FeedPP::Atom->new();
    $feed->merge( "rss-2.xml" );     # RSS 2.0
    $feed->merge( "rss-1.rdf" );     # RSS 1.0 (RDF)
    $feed->merge( "atom-1.xml" );    # Atom 0.3/1.0
    
    my $item = $feed->add_item( "http://search.cpan.org/~kawasaki/XML-TreePP-0.02" );
    $item->title( "Pure Perl implementation for parsing/writing xml file" );
    $item->pubDate( "2006-02-23T14:43:43+09:00" );
    
    $feed->to_file( "index.rdf" );
    

    RSS 2.0, RSS 1.0 (RDF), Atom 0.3 and 1.0 are supported.

    §3
    Web services in Japan

    Recruit provides 14 kinds of APIs

    Open data APIs for EC, car, local search, human resource, gourmet coupon, traveling, school, etc.
    http://webservice.recruit.co.jp/

    Mashup Award 4th

    A largest contest for web application development in Japan.
    1st prize: JPY¥1,000,000! (≒US$10K) http://mashupaward.jp/english/

    38 Cosponsors (100+ APIs)

    Most of web services are based on POX.
    (POX = Plain Old XML)

    <results>
    
      <api_version>1.00</api_version>
      <results_available>8629</results_available>
      <results_returned>10</results_returned>
      <results_start>1</results_start>
      <usedcar>
    
      <id>CU0001829778</id>
      <brand>
        <code>LE</code>
        <name>LEXUS</name>
      </brand>
      <model>SC</model>
    
      <grade>430</grade>
      <price>5300000</price>
      <desc>HDD navigation TV</desc>
      <body>
        <code>O</code>
    
        <name>Open</name>
      </body>
    

    Recruit Web Service UI Library

    JavaScript libraries to handle user interface elements on websites.

    Mashup Example

    Google Maps API+Hotpepper Web Service
    It took just two hours to read spec sheets and write this code!

    UI Library Usage

    <script type="text/javascript" src="js/jquery-1.2.3.js"></script>
    <script type="text/javascript" src="rui/recruit.ui.js"></script>
    <script type="text/javascript" src="rui/hotpepper.ui.js"></script>
    <script type="text/javascript">
        function hpp_init (){
            Recruit.UI.key = '********';
            new HotPepper.UI.Places.Pulldown();
            new HotPepper.UI.Genre.Pulldown();
        }
        $(hpp_init);
    </script>
    <form id="hpp_sample" action="#">
    Area: <select id="hpp-large-service-area-sel"></select>
    <select id="hpp-service-area-sel"></select>
    <select id="hpp-middle-area-sel"></select>
    <select id="hpp-small-area-sel"></select><br>
    Food genre: <select id="hpp-genre-sel"></select>
    <input type="submit" value="Search" />
    </form>
    

    §4
    WSST

    (Q) How long do you need to write a Perl module? (in general)

    2 months?
    2 weeks?
    2 days?

    * Ingy took about only two hours to write Vroom.pm for OSDC.TW in Taipei, but it's an exception.

    Our solution

    WSS - WebService Specification Schema based on YAML

    WSST - Tools to generate well tested Perl module by WSS

    WSST needs only two seconds to write a Perl module!

    Write once, get multiple libraries

    In addition to Perl modules, we (will) support PHP, Python, Ruby, ActionScript etc. by one schema.

    How many libraries required?

    14 web services
    X
    5 languages
    ||
    70 libraries (!?)

    It's difficult for us to keep maintaining such large numbers of libraries by our hands!
    Auto-library-generation is good way to do.

    Perl module usage

    use WebService::Recruit::CarSensor;
    
    my $service = WebService::Recruit::CarSensor->new();
    my $param = {
        'key' => 'XXXX',
        'pref' => '13',      # Tokyo
    };
    my $res = $service->usedcar( %$param );
    my $root = $res->root;
    my $list = $root->usedcar;
    
    foreach my $car ( @$list ) {
        print "Model: ", $car->model, "\n";
        print "Brand: ", $car->brand->name, "\n";
        print "Price: JPY ", $car->price, "\n";
        print "Shop:  ", $car->shop->name, "\n\n";
    }
    

    CarSensor web service provides market prices of used cars.

    PHP library usage

    <?php
      require_once 'Services/Recruit/CarSensor.php';
      $service = new Services_Recruit_CarSensor(array('key' => 'XXXX'));
      $res = $service->usedcar(array('pref' => 13));
      $usedcar = $res->getData();
    ?>
    
    <?php foreach($usedcar->usedcar as $i) { ?>
    <table>
    <tr><th>Model</th>
    <td><?php echo htmlspecialchars($i->model) ?></td>
    
    </tr>
    <tr><th>Price</th>
    <td>JPY <?php echo htmlspecialchars($i->price) ?></td>
    </tr>
    
    </table>
    <?php } ?>

    WSST is now on CPAN

    http://search.cpan.org/dist/WSST/

    Diagram

    WSS (YAML) example

    company_name: Flickr
    service_name: Test
    version: 0.0.1
    methods:
      - name: echo
        title: flickr.test.echo
        url: http://api.flickr.com/services/rest/
        params:
          - name: method
            require: true
            fixed: flickr.test.echo
          - name: api_key
            require: true
        return:
          name: rsp
          children:
            - name: stat
            - name: method
            - name: api_key
     
    

    WSS (YAML) example, cont.

        error:
          name: rsp
          children:
            - name: stat
            - name: err 
              children:
                - name: code
                - name: msg
        tests:
          - type: good
            params:
              api_key: $FLICKR_API_KEY
          - type: error
            params:
              api_key: invalid_api_key
     
    

    Templates files for Template.pm

    perl/lib/WebService/{company_name}/inc_webservice_method.tmpl
    perl/lib/WebService/{company_name}/{service_name}/Base.pm.tmpl
    perl/lib/WebService/{company_name}/{service_name}/{method.class_name}.pm.tmpl
    perl/lib/WebService/{company_name}/{service_name}.pm.tmpl
    perl/make-dist.sh.tmpl
    perl/Makefile.PL.tmpl
    perl/README.tmpl
    perl/t/00_pod.t.tmpl
    perl/t/01_new.t.tmpl
    perl/t/02_new_methods.t.tmpl
    perl/t/03_setget.t.tmpl
    perl/t/inc_test_webservice_method.tmpl
    perl/t/{test_seq}_Method_{method.class_name}.t.tmpl
    perl/t/{test_seq}_{service_name}.t.tmpl
    

    {method.class_name}.pm.tmpl

    package [% package_name %]::[% method.class_name %];
    
    use strict;
    use base qw( [% package_name %]::Base );
    use vars qw( $VERSION );
    use Class::Accessor::Fast;
    use Class::Accessor::Children::Fast;
    
    $VERSION = '[% version %]';
    
    =head1 NAME
    
    [% package_name %]::[% method.class_name %] - [% title %] "[% method.name %]" API
    
    =head1 SYNOPSIS
    
        use [% package_name %];
        
        my $service = [% package_name %]->new();
        
        my $param = {
    [% FOREACH key = sort_keys(method.first_good_test.params) -%]
            '[% key %]' => [% method.first_good_test.params.$key | $env_param %],
    [% END -%]
        };
        my $res = $service->[% method.interface_name %]( %$param );
        my $root = $res->root;
    [% FOREACH node = tree_to_array(method.return) -%]
    [% LAST IF loop.index > 6 -%]
    [% NEXT IF node.depth == 1 -%]
        printf("[% node.name %]: %s\n", $root[% node_access(node) %]);
    [% END -%]
        print "...\n";
    

    Generating Perl module

    $ wsst generate flickr-test.yml -l perl -o output
    [perl]
    output/perl/lib/WebService/Flickr/Test.pm
    output/perl/lib/WebService/Flickr/Test/Base.pm
    output/perl/lib/WebService/Flickr/Test/Echo.pm
    output/perl/lib/WebService/Flickr/Test/Login.pm
    output/perl/Makefile.PL
    output/perl/README
    output/perl/make-dist.sh
    output/perl/t/00_pod.t
    output/perl/t/01_new.t
    output/perl/t/02_new_methods.t
    output/perl/t/03_setget.t
    output/perl/t/04_Test.t
    output/perl/t/05_Method_Echo.t
    output/perl/t/06_Method_Login.t
    

    It takes roughly two seconds to generate a module.

    Generating PHP library

    $ wsst generate flickr-test.yml -l php -o output
    [php]
    output/php/package.xml
    output/php/Test.php
    output/php/Test/Factory.php
    output/php/Test/Method/Echo.php
    output/php/Test/Method/Login.php
    output/php/Test/Method.php
    output/php/Test/Request.php
    output/php/Test/Request/Default.php
    output/php/Test/Response.php
    output/php/Test/Response/Default.php
    output/php/docs/examples/echo.php
    output/php/docs/examples/login.php
    output/php/tests/AllTests.php
    output/php/tests/Services/Flickr/Test/FactoryTest.php
    output/php/tests/Services/Flickr/Test/Method/EchoTest.php
    output/php/tests/Services/Flickr/Test/Method/LoginTest.php
    output/php/tests/Services/Flickr/Test/Request/DefaultTest.php
    output/php/tests/Services/Flickr/Test/RequestTest.php
    output/php/tests/Services/Flickr/Test/Response/DefaultTest.php
    output/php/tests/Services/Flickr/Test/ResponseTest.php
    output/php/tests/Services/Flickr/TestTest.php
    

    WebService::Simple::Cabinet

      use WebService::Simple::Cabinet;
      my $flickr = WebService::Simple::Cabinet->new(
          {
              global => {
                  name     => 'flickr',
                  package  => 'Flickr',
                  base_url => 'http://api.flickr.com/services/rest/',
                  params   => {
                      api_key => undef,
                  },
              },
              method => [
                  {
                      name   => 'echo',
                      params => {
                          method => 'flickr.test.echo',
                          name   => undef,
                      },
                      options => {},
                  },
              ],
          },
          api_key => 'your_api_key',
      );
      my $res_xml = $flickr->echo( name => 'echo data' );
      print $res_xml->{name};
    

    is yet another solution developed in CodeRepos.

    Acknowledgements

    Thank you.

    Feel free to contact me:
    u-suke [at] kawa.net
    http://www.kawa.net/