Scheme/lisp require a totally different way of thinking to solve problems.

I was just browsing the

net

 .
One guy's comment why scheme sucks is because it does not have resizable arrays/vectors.
This comment is 100% true.
Obviously he does not understand functional

programming

  vectors/arrays are not needed.
It is like saying that a racing car sucks because it does not have pedals like a bike has.

Simple question to ask, can you implement a

python

  generator in language X.
Lets say that you like smalltalk objects (i have no idea what they are)
Could you implement that in Language X.
I have asked people that question hundreds of times.
They hum and haw, well language X allows you to access assembler.
A better question to ask is can you go into your language X's compiler and modify the parser and the way  it generates code ?
Except for dylan,ocaml,lisp,scheme and possibly

erlang

  (i am not 100% sure) most likely the answer is no.

Anyway this guy shows u how to do it in scheme

Origional version of this document is here.
I cut and paste from G cache here because quite often this guy's server is down.
http://xmog.com/scrap/show/5

<>Why Java (and almost every other

programming

  language) sucks

Ed Watkeys, edw@xmog.com
September 8, 2005
Updated December 27, 2005

One of the big new features of Java 5.0 is a new syntax for iterating over collections. Instead of tediously typing the following:

for (Iterator i = c.iterator(); i.hasNext(); ) {
    String s = (String) i.next();
    ...
}

you can now, thanks to the fabulous new iteration syntax along with the introduction of generic types, substitute the following code:

for (String s : c) {
    ...
}

That’s clearly a lot less keyboard pounding. But a question begs to be answered: Why did it take ten years to introduce this feature? Let’s forget about that important question for now and look at another

programming

  language,

Python

 .

Python

  is in many ways a much nicer language to program in. And it evolves more quickly than Java. For example, generators were introduced in

Python

  2.2. A generator is a function that can produce multiple values, maintaining state between each call. Here’s a simple example:

def counter(n):
  while True:
    yield n
    n = n + 1

Because this function definition contains the keyword yield,

Python

  knows it’s a generator. You use the counter generator like this:

c12 = counter(12)
c12.next()
c12.next()

The first line creates an instance of the generator that starts counting at twelve. The second line tells the generator to run until it yields a value. The third tells the generator to resume running until it yields another value. The first two values yielded by this generator are the integers twelve and thirteen.

This is a cool feature: It lets programmers write simple code that without generators would be complex and error-prone. Why can’t Java be more like

Python

 ?

Lets put this second question aside and think about how me might implement

Python

 ’s generators in a language used by some of the most smug weenies in the world, Scheme. Scheme is a dialect of Lisp that’s been around in one form or another for about thirty years. Lisp has been around in one form or another for almost fifty years.

Here’s the best Scheme implementation I could come up with that works like the

Python

  counter generator:

An aside for experienced Lisp programmers: The procedure I’m about to show you is far more complicated than the canonical example of an accumulator, because I’m duplicating the semantics of

Python

 ’s generators.

(define (counter n)
  (letrec ((generator
            (lambda (yield)
              (let counter ((n n))
                (call-with-current-continuation
                (lambda (continue)
                  (set! generator (lambda (k)
                                    (set! yield k)
                                    (continue n)))
                  (yield n)))
                (counter (+ n 1))))))
    (lambda () (call-with-current-continuation
                (lambda (yield)
                  (generator yield))))))

“OMFG!” you must be saying to yourself. OMFG indeed! In the original version of this scrap, I said writing this wasn’t so so difficult. I then found a bug that would lead to an infinite loop when the counter was used in certain non-trivial ways. So I’m going to come out and admit it: People shouldn’t have to write procedures like this if they simply want to write a function that acts like a

Python

  generator. On the plus side, this monstrosity can be used very simply by client code:

(define c12 (counter 12))
(c12)
(c12)

The first line defines c12 to be the result of the procedure counter called with twelve as its sole argument. The second and third lines call c12 with no argument and return, just like the

Python

  examples, the values twelve and thirteen. But this is all academic, because no sane person would write procedures like this on a regular basis.

Writing procedures like counter regularly leads to cramped fingers and a head ready to explode. But it’s interesting to note that it is possible to write counter, and that, to the outside world, the Scheme generator is easier to use than the

Python

  version, because the Scheme version returns a procedure, which can be called like any other procedure, unlike

Python

  generators, which return generator objects, which require programmers to call the next method.

(An aside: The designers of

Python

 ’s generators could have opted to implement generator objects in such a way that the next value could be retrieved via c12() and c12.next(), but they didn’t. The decision doesn’t make any sense to me. And while I love many things about

Python

 , there’s a sort of ugliness that pervades the non-trivial corners of the language.)

Back to Scheme… The error-prone tedium of writing these generators in Scheme would seemingly make them impractical, but they’re not, because Scheme includes a feature that

Python

  and Java lack: the ability to extend the syntax of the language. If you can manage to write the Scheme version of counter, it isn’t much more effort to create a

mac

 ro that makes this feature available in an accessible way. Here’s the

mac

 ro code I wrote that does just that:

(define-syntax define-generator
  (syntax-rules ()
    ((define-generator (NAME ARG ...) YIELD-PROC E1 E2 ...)
    (define (NAME ARG ...)
      (letrec ((generator
                (lambda (yield)
                  (let ((YIELD-PROC
                          (lambda v
                            (call-with-current-continuation
                            (lambda (continue)
                              (set! generator (lambda (k)
                                                (set! yield k)
                                                (apply continue v)))
                              (apply yield v))))))
                    (let NAME ((ARG ARG) ...)
                      E1 E2 ...)))))
        (lambda () (call-with-current-continuation
                    (lambda (yield)
                      (generator yield)))))))))

Once you have this

mac

 ro, the Scheme version of the counter generator looks like this:

(define-generator (counter n) yield
  (counter (+ 1 (yield n))))

Not bad, eh? The only thing that bothers me about this version is that I need to specify the name of the yield procedure. But one could argue that it gives programmers flexibility to give the procedure whatever name make most sense given the context of the code. (Again, experienced Lispers will know that this “feature” could be fixed by using non-hygenic

mac

 ros, but we’re sticking to standard, R5RS Scheme here.)

If you compare the first and second versions of counter, you might notice that I did something tricky in the new, define-generator version: the yield procedure returns the value that it yields, so it can be used in the recursive call to counter. And you can’t do that with

Python

 ’s generators.

So why can’t Java be more like

Python

 ? The answer is Java is a lot like

Python

 :

Python

  users had to wait around for about ten years before they got generators. I added support for generators in Scheme in a few hours of playing over three days. We can argue that generators, as well as other recent features of

Python

 , like list comprehensions, make

Python

  a more pleasing language to work in — and I wholeheartedly agree with that argument — but fundamentally, Java and

Python

  are alike in that you can’t modify the language itself.

Java,

Python

 , and nearly every other non-Lisp language in existence put you at the mercy of language designers. You need to wait for them to implement the language features at the top of your wish list. And when they do manage to scratch your itch, who’s to say you’ll like the result?

So why did it take ten years for the enhanced iteration syntax to make its way into Java? It took so long because in Java, as in most other

programming

  languages, syntax is a big deal. You just don’t go changing a language’s syntax. It’s hard to do, and only a select few have the skills to do it. And when it happens, expressiveness and clarity take a back seat to preserving backwards compatibility.

In Scheme, adding syntax is relatively easy, and can be done on a per-problem basis, so you don’t have to worry about coming up with the ideal-for-all-time solution. This ability to build the language up to a problem trumps any concern over writing in a language that uses a lot of parentheses.
Links

Schemers.org
Scheme resources
Lisp resources

Python

  Generators in Ruby
Copyright 2005-6 Transmogrify, LLC. All rights reserved.

A few points to add.
I would not recomend a newbie to

programming

  to

learn

  scheme/lisp,

learn

  something like

python

  or

php

  which are user friendly.

Also if you are the type who likes community help etc, probably not a good language for you ,
Documentation is limited, and unlike other languages there are countless implementations floating arround.

I will add more resource and stuff later.

Great post Nop

I am curious what makes Scheme/Lisp so attractive to you?

i did scheme/lisp in college at some point and those parens freaked me out.

my question would be, the short version of what scheme's advantage is. Just curious. Im probably never gonna do scheme, but you never know.

at least from a web development point of view I get in this conversation all the time regarding languages. Im an

ASP

  guy, and I still have not run into something I cant do that others can in their language of choice. (elegance and libraries aside)

PHP

  for example I always hear "there are so many libraries available". For me, thats the problem lol. I cant remember any of the functions/classes. Thats been my biggest problem with shifting to an

ASP

 

.net

  language like C#. The stupid namespaces cause me way to much grief. as well as the never ending Dot-notations. (http.what.freaking.method.do.i.use.to.string.concat)

That's really the point of my question here as well -

There are loads of beautiful libs in any language you go to... and of course, they'll be most beautiful to anyone working in that language that is looking for <that> functionality...

So I'll use my

PHP

  decision here as a "why" for my question - (in short) I chose

PHP

  because it is OS and server agnostic, a reasonably normal syntax, great amount of future (dead languages suck) can be compiled and cached like Java,

ASP

  etc and is NOT complicated by a huge morass of crap like EJB frameworks. I was stung by dead languages with Delphi/Kylix (Object Pascal) and I have become very unenamoured with huge-project compiled languages - the single script notion works nicely for me in the Web world.

So what gives you a woody abut Scheme/Lisp?

The syntax of scheme is ugly .
Hard to find proper docs.
Because so many different variants, and ways to do thinks libs are very hard to find.
Oh i thought u where talking about bad parts of scheme

The power is the

mac

 os, closest example is meta-

programming

  in

python

 .
Or code that generates code.
They are not libs.

Everythink in scheme/lisp is a list.
The difference between data and code is not really there.
reference
http://www.ccs.neu.edu/home/dorai/t-y-scheme/t-y-scheme.html
I will steal example from this book since he do better job explaining then me.
chapter on

mac

 ros http://www.ccs.neu.edu/home/dorai/t-y-scheme/t-y-scheme-Z-H-10.html#node_sec_8.1
scheme does not have a "while" statement like

python

 ,

php

  etc have. (because you should use recursion very bad to make a while).
you decide you want one.


that little chunk of code allows u to have a while statement

so you can now do stuff just like ur

python

 ,

php

 ,C friends


the possibilities are endless.
when wrapping libcurl (my libcurl is not the same as one chicken uses).
Anyone who make binding know that wrapping is repeditive boring and prone to errors.

in lib curl you have a call like this.
CURLcode curl_easy_setopt(CURL *handle, CURLoption option, parameter);
it sets the options on the curl.
CURLoption is some sort of enum ie CURLOPT_URL,CURLOPT_PROXY the parameter can be either a int,string,float,list etc.
There are like lots of them

Anyway i make a

mac

 ro that look like this (just showing header)
(define-

mac

 ro (

mac

 ro-set-opt name option type) I added option type where i stuff in there string,int etc.
depending what the type

mac

 ro generates appropriate code.
(

mac

 ro is like 20 lines)

Then i just go like this.
(

mac

 ro-set-opt url CURLOPT_URL __std::string)
(

mac

 ro-set-opt connect-timeout CURLOPT_CONNECTTIMEOUT int)
(

mac

 ro-set-opt httpheader CURLOPT_HTTPHEADER __slist)
......
the code is generated that generates the approriate binding.
My binding is 800 lines counting a complex state-

mac

 hine and includes interface to multi which

PHP

  did not include to version 5.
(I am not sure how big

PHP

  binding is I bet more then 800 lines )
Lets say i have memory leak when i am handling strings, to go back and fix very easy.

The possibilities are endless.
You can make your own object system.
You own way of records etc.

-------------------------------------------------------------------------------------------------------------------
The downside.
I can make a language that i like.
The consequences is that group projects in scheme/lisp are almost impossible.
(you can argue they are not needed)
But you as a program manager are in big danger.
You hire joe to make a webserver.

Joe does good job, builds webserver in 2 weeks.
Joe uses lots of complex

mac

 ros etc server is only 10k lines of code.
You piss joe off and he quits. You are fuked

So Scheme fits the way you think... I totally get it. I also see why this is NOT the language/structure for collaborative projects - good lord what a mess that would be.

All that being said, after reading your examples about 3 times and trying to wrap my head around it I have developed a migraine and accute tick in my left eye. Back to simple procedural/OO languages for me 

It is like the movie Mr Baseball.
In the movie the woman saids "Everything that is japanese is the best"
Mr Baseball saids "That is impossible everything that is japanese is the best ?"
"I will prove it to you, I will make you japanese meal, and you will agree that it is the best"
She disap

pear

 s for a hour and then comes back with a tray, on the tray there is an american steak and potatoes.
"Here is best japanese meal".
Mr Baseball "No that is american meal"
"No all things in the world that are the best we make japanese, steak and potatoes japanese meal" 

All

programming

  features that are the best are made into scheme.
Scheme is both compiled and interpreted 

The idea is very simple.

Python

  has decorators,generators etc.
Lets say as a

PHP

  programmer you wished you had those things. Basically all you can do is hope one day in the future maybe

PHP

  will adopt it.

As a scheme/lisp programmer I can take that feature, and implement it as part of the language right now.
What ever new OO system language X has implemented I can implement in scheme if i desire.
I can program scheme functionally or procedurally. It is extremely complex but very simple everything is a list 

I am curious if that serves you better in the web/spam world than a procedural language - I can definitely see how, in a place where you need to create new metholodolgies or think in unconventional ways that this would serve you immensely... but how does that thinking, or more precisely, that level of abstraction and complexity make <>spamming guest books creating great web sites easier?

I'm also curious if you ever touched Prolog Nop - that's another strange language - I had real trouble with that when I first played with it in the mid 80s.

I looked at prolog but I never did anything with it, ironically it was prolog that kinda set things off
I looked at prolog but i could see nothing practical to use it with
It was a couple years back, I had been using

python

  extensively.

I started using sqlobject, If you do database apps in

Python

  http://www.sqlobject.org/
In a nutshell It allows you to describe database tables as what ap

pear

 s to be a

python

  class.
Or it can take a database table and map it to a class.

I later on

learn

 ed this magic was performed by something called metaclasses.
(basically classes that generate classes).
sqlobject is a prime example where metaclass needed

Anyway i ran into prolog, from there i looked at ocaml, and functional

programming

 .
Initially i thought scheme was a stupid language.
All of the fuking brackets.
What type of moronic language does not have an array ?
In the meantime still using

python

 . (I still use

python

 , great for many things).

Then someone posted on the

net

  some free

programming

  videos from MIT
and a link to a free book.
http://mitpress.mit.edu/sicp/full-text/book/book.html
I started to look at the book.
Ofcourse i never read book from start i look at back.
WTF you can build a scheme compiler in scheme that is less then 100 lines.
It makes no sense but ok maybe i should look into this shit.
Go install fuking e

mac

 s, who the fuk invented this wacked out editor.

Anyway I installed plt scheme and started to go thru example in book.
http://www.plt-scheme.org/software/drscheme/
A good scheme for newbies.

Later on I switch to chicken scheme.
Because you can call C from scheme directly.

Anyway i create a libcurl binding.
But to connect to many sockets on one thread you need to use select.
(libcurl uses this internally)
Reading many sockets from one thread causes a multitude of head-ache complexity.

Where does scheme come to the rescue here.
I make a

mac

 ro that looks similar to the state

mac

 hine

mac

 ro except that it handles sockets.
example
one state ---- get google stuff
(get-url curl
            (format "http://www.google.com/search?q=~a&num=100&safe=off&start=~a&sa=N&filter=0"
                    (curl-escape query) offset))
    (add-curl multi curl)
    (<-- search-result offset) <><--- when socket has results go to state search-result, if timeout error attempt to do again
    )
 
  (search-result <><--- result if here
    (offset)
    -> do something with
-----------------------------------------------------
In the big part I have what I call an engine.
The engine processes the sockets, checks when they are ready etc.
each socket has a state

mac

 hine on it.

In a regular language you would end up with a mess .
But all this is hidden by the

mac

 ro. so all u see is above.

As a result on one thread I can have 20 open sockets .
Less threads = more performance on a VPS

The engine could be fed a G scrapper, a MSN scrapper, a Y scrapper and they all will
keep track of them selves. Since each one ap

pear

 s to be an individual

complete G scrapper for reference

I *think* I get it.

(BTW - please post the G Scraper in the Code Repository - that's great)

I am intrigued intellectually, although I cannot make a case for myself yet why I would go any deeper.

I am going to reread your posts and the link to the school site because this is a pretty radically different way of thinking for me. Amoun777 used to harrass me because he'd go have a good time somewhere and I wouldn't join him because I'd be knee deep reading "Common Graphical Formats and Specifications" or something. (I still have that book, BTW  ) so I will work towards grokking what you are saying.

G scrapper is useless without the

mac

 ro.
Oneday i will clean up the

mac

 ro and publish it.

No reason for u to go deeper.
In the big shops instead of coding smarter, solution is to divide problem up into smaller pieces and hire more coders.
You praise pascal (delphi) because of its strict typechecking etc. If you are running a project with 10-40 guys working on it
yepper this is good stuff . You stated in another thread OO makes your code bigger.
If you have 10-40 code monkeys who cares 

But on the other hand if there is just 1 person, strict typechecking, bloated code is a bad thing. I do not have the luxary of
hiring extra people.

Yahoo Stores is a prime example. Origionally developed in lisp by one man.
Bought out by yahoo. Then entire thing rewritten in some language like

php

  with probably 40+ guys.
If i was running yahoo i would have done same thing

Nop you amaze me.

That wouldn't've been me... I see the organizational and maintenance benefits of OO as FAR outstripping the < relatively tiny > code increases... I'm an OO man from the moment I get up in the morning.

im a bOOb man also.

Yeah baby!

Pink hat struts her stuff:

w00t

heh 

you, not you individually but the <>Imperial You, programmers and language developers are in serious need of engineers. hmm prolly mechanical engineers.

<>added: hee thanks i type like a cabbage

you all need something  like a Carnot cycle and efficiency rating for languages imo

imperative you? I don't think we're imperilled here...