The Cache: Technology Expert's Forum
 
*
Welcome, Guest. Please login or register. September 18, 2019, 12:33:51 AM

Login with username, password and session length


Pages: [1]
  Print  
Author Topic: Interesting experience with Snow Leopard's loading features  (Read 2126 times)
kurdt
Lifer
*****
Offline Offline

Posts: 1153


paha arkkitehti


View Profile
« on: September 17, 2009, 03:26:37 AM »

Ok, so I have this 450MB text file I want to open. Let's open it with Textmate.. after 30 secs -> out of memory and empty file opens. Textmate doesn't crash however, it just opens the file as empty. Let's try with TextEdit. Now you have to know that TextEdit is suppose to be all Snow Leopard compatible and should take advantage of 64bit, GCD and other speed up features Snow Leopard has to offer.

Now interesting thing happened. First, I had only about 100MB free memory so it consumed that and then it started to move memory around and it was loading into RAM like a dream. However at this point I noticed that 1 core (My MacPro has 8 cores) was working 100% processing TextEdit. Interesting thing is that it never utilized anymore than that 1 core. It didn't expand to other cores at all. I monitored I/O and disk usage and there was next to none so bottleneck wasn't harddrive->cpu channel. Everything was loaded into RAM at this point but nothing happened. After 3 minutes TextEdit opened the file and everything was just fine.

Even it did exactly what I wanted I'm still little confused about the behavior. Wasn't this whole Snow Leopard and revamping old programs suppose to utilize multicore processors better? I mean I always thought that was the goal with all this GCD stuff.
Logged

I met god and he had nothing to say to me.
perkiset
Olde World Hacker
Administrator
Lifer
*****
Offline Offline

Posts: 10096



View Profile
« Reply #1 on: September 17, 2009, 09:43:33 AM »

Yes, SL does a great job of multiprocessing, considerably better even than Leopard. The issue is TextEdit and the task at hand, not SL.

Consider for a moment how you'd rewrite a text editor to load a single file stream, loaded serially, into a memory buffer. The complexities of trying to do that as many different little jobs across multiple cores would be silly and not provide any benefit. Where it really screams is with things that are built for multi-core, like Handbrake or MTR. Also, with SL, one core can be totally dominated by the TextEdit task and still have lots of processor left for other stuff that needs to run. Now: a really smart text editor might start loading and let you edit the first page while it's paging the rest, but that introduces a new set of challenges. And after the initial buffers have been built, there is work to do to prepare the pages for "dirty notification" and such that could be multi-cored, but the benefits would be outweighed by the costs, I'm thinking.

You might try BBEdit or TextWrangler, which I don't think would have the same troubles as TextEdit - that little program is akin to Notepad in Windows and was really not designed for such work. TextWrangler (Bare Bones Software) is the scaled-down version of BBEdit (what I use for all my coding) and *is* designed for big file editing/manipulation - but I still don't know how long it would take for it to put it all up into pages of RAM for you to start editing.

The promise of multicore is only partially, and probably a minority, for applications. Some apps, like Safari particularly, make stellar use of multicore because once you've downloaded the HTML and CSS for a page, then you can have cores doing JS, cores getting graphics, cores rendering portions of the page etc etc... but for the most part, computer "jobs" tend to look more like word processing - where there's a single point of focus (where you're cursor is and what you're doing) ergo, really, a difficult hike trying to multi-core the app because we can't take our notion of "the job" and divide it into lots of discreet little tasks. But the benefit is that you can be scraping videos, playing music, word processing while you have 20 browser tabs open, your contacts, date book automatically updating from the cloud and a TV streamer showing you Dr. Phil at the same time... with minimal, if any, degredation in speed. That is precisely how my system operates, and why I have 6 monitors and 8 spaces. I have boatloads of things open and running *all the time.* The multicoreness of the box makes it so that it can still keep up with me (a feat, to be sure) even when I've got a whole lot going on.

All that said, everyone is working on multicoring and the new libs from Apple will really start to promote this: and as soon as people start really thinking in terms of jobs as little pieces rather than jobs a monolithic processes, we'll see a revolution in the way apps are written once again (this is across all platforms, the Mac's new multicore standard is just the sharp end of the stick).
Logged

It is now believed, that after having lived in one compound with 3 wives and never leaving the house for 5 years, Bin Laden called the U.S. Navy Seals himself.
kurdt
Lifer
*****
Offline Offline

Posts: 1153


paha arkkitehti


View Profile
« Reply #2 on: September 17, 2009, 11:04:43 AM »

Yeah you are right and I did realize that it was Textedit that was the faulty part but it was still disappointment because I guess I expected that SL could some how balance the loading by itself. I guess I just expect too much from operating system. It's just funny how still everything is tied to the app itself. Simplified example what I'm after could be that if you have operating system that runs a compiled code it could dispatch first calculation pair to 1st core, second pair to 2nd core and so on... and now you have to also understand that I know shit about assembly coding or how OS actually runs programs so all these things are just what I would wish could be possible.
Logged

I met god and he had nothing to say to me.
perkiset
Olde World Hacker
Administrator
Lifer
*****
Offline Offline

Posts: 10096



View Profile
« Reply #3 on: September 17, 2009, 11:21:03 AM »

...that if you have operating system that runs a compiled code it could dispatch first calculation pair to 1st core, second pair to 2nd core and so on... and now you have to also understand that I know shit about assembly coding or how OS actually runs programs so all these things are just what I would wish could be possible.

ASM is the same as PHP is the same as anything else (some caveats to that statement for languages like Prolog, LISP etc, but for now, good enough). The processor doesn't really do much of anything at all, except execute binary decisions on lots of lots of transistors as the assembly tells it to do. A processor would have no notion of *why* to divvy up a program and say do this here, do this there, because the processor would not know if step B was dependent on step A or not. If the processor/OS was smart enough to do that, then it is an interpreter, not just an OS or processor. In fact, the processor/OS will simply see steps that need to be executed in order ... it shouldn't have an inherent skill at dividing tasks or you've burdened the processor/OS with too much smarts and that will slow overall performance.

Divvying is the job of the developer. The developer needs to be able to logically see a job as many parts, then tell the processor to execute the parts in concurrence. That's how the really big software that renders movies works, for example. If you consider a single frame of a movie a job, and it takes 2 hours to completely render the frame, then at 30FPS, a 2 hour movie would take just two hours to render ... if you have 216,000 processors - 2s compliment those to get the net time for less ie., 1/2 the processors, double the time. But jobs like "bold from here to there" in a document cannot easily be divvied. Ergo, no time is even spent on it because the payoff would be so low.

That said, there is already work done along this line. Note how you can be typing and there's red underline under a mis-spelled word. That's a classic example of the spelling daemon running concurrently to the GUI daemon and the cursor daemon and the keyboard watcher. So there are efforts along this line.

The real bottleneck is us - we don't naturally see jobs as huge concurrent multitasks. We see serialization well, and computers thus far have  mapped our perception of the world. This is changing, as people's brains grow to assimilate the possibilities of massive parallel processing. There is talk of a 16 core iPhone for example. Thousands of cores on desktop dies. What needs to grow, is our ability to do many things at once.
Logged

It is now believed, that after having lived in one compound with 3 wives and never leaving the house for 5 years, Bin Laden called the U.S. Navy Seals himself.
nutballs
Administrator
Lifer
*****
Offline Offline

Posts: 5627


Back in my day we had 9 planets


View Profile
« Reply #4 on: September 17, 2009, 11:33:51 AM »

try dragging a 2GB PSD into notepad in windows. BOOM lol
Logged

I could eat a bowl of Alphabet Soup and shit a better argument than that.
kurdt
Lifer
*****
Offline Offline

Posts: 1153


paha arkkitehti


View Profile
« Reply #5 on: October 05, 2009, 05:14:24 AM »

I just opened 5.77GB text file with TextEdit and it actually worked Cheesy It took few minutes to open up but it finally did. That's just nice Smiley I might try that PSD to see if I can get my SL to go BOOM.
Logged

I met god and he had nothing to say to me.
nutballs
Administrator
Lifer
*****
Offline Offline

Posts: 5627


Back in my day we had 9 planets


View Profile
« Reply #6 on: October 05, 2009, 10:09:31 AM »

lol thats actually pretty impressive. Was it just ascii text?

I think because PSDs are binary that totally jacks editors like notepad maybe.
I do it all the time by accident.
Logged

I could eat a bowl of Alphabet Soup and shit a better argument than that.
Pages: [1]
  Print  
 
Jump to:  

Perkiset's Place Home   Best of The Cache   phpMyIDE: MySQL Stored Procedures, Functions & Triggers
Politics @ Perkiset's   Pinkhat's Perspective   
cache
mart
coder
programmers
ajax
php
javascript
Powered by MySQL Powered by PHP Powered by SMF 1.1.2 | SMF © 2006-2007, Simple Machines LLC
Seo4Smf v0.2 © Webmaster's Talks


Valid XHTML 1.0! Valid CSS!