The Cache: Technology Expert's Forum
 
*
Welcome, Guest. Please login or register. September 17, 2019, 06:11:58 PM

Login with username, password and session length


Pages: [1]
  Print  
Author Topic: Hi first post... replacing words in a chunk of text (after filtering stopwords)  (Read 1743 times)
herbacious
Journeyman
***
Offline Offline

Posts: 51


View Profile
« on: October 05, 2007, 09:50:50 AM »

as requested by the big P.. (originally posted on Syndk8)

here is my code at the moment - i am finding it is a bit slow and would love to make it faster and more accurate if possible.

Code:
<?php
$text 
'blah blah keywords blah, etc bollocks weather shit winter dark blah buy something etc blah and this stopword ho hum';

$linkified linkification($text);

/*note linkification table has words and associated links

WORD       LINK
bollocks : http://www.bollocks.com
keywords : http://keywords.buy-from-me.com
*/

function linkification($text){
$words strip_tags($text);
$words ' ' trim($words) . ' ';
$words preg_replace("([[:punct:]])",'',$words);
$words explode(' '$words);
$stopwords stopwords();
/* perkiset says this is not good
foreach($stopwords as $sword){
$found = array_search($sword, $words);
if($found){
unset($words[$found]);
}
}
*/
foreach($words as $word){
if (array_search(strtolower($word), $stopwords) === false){
$clean_words[] = $word;
}
}
$words implode("' || word = '"$clean_words);
$sql "select distinct * from linkify where ( word = '" $words "')";
$links_query mysql_query($sql);
while($link mysql_fetch_assoc($links_query)){
$text = linkify($link['word'],$link['link'],$text);
}
return $text;
}


function 
linkify($lookfor$link$text$case true){
if($case){
$output preg_replace('%( )' $lookfor '( )%i',' <a http://' $link '">\\0</a> '$text);
}else{
$output str_ireplace(' ' $lookfor ' '' <a http://' $link '">' $lookfor .'</a> '$text);
}
$output trim($output);
return $output;
}


function 
stopwords(){
$stopwords = array('when''effectively','anti','to''a''and','get','into''with''been''what''with''also''that''into''a''all''an''and''are''as''at''be''but''by''can''for''from''had''have''he''her''his''in''is''it''may''not''of''on''or''that''the''this''to''was''which''will''with''you''your''more''they''people''want''last''2007''even''simply''better''massive''exhibition''a''able''about''above''abroad''according''accordingly''across''actually''adj''after''afterwards''again''against''ago''ahead''ain\'t''all''allow''allows''almost''alone''along''alongside''already''also''although''always''am''amid''amidst''among''amongst''an''and''another''any''anybody''anyhow''anyone''anything''anyway''anyways''anywhere''apart''appear''appreciate''appropriate''are''aren\'t''around''as''a\'s''aside''ask''asking''associated''at''available''away''awfully''b''back''backward''backwards''be''became''because''become''becomes''becoming''been''before''beforehand''begin''behind''being''believe''below''beside''besides''best''better''between''beyond''both''brief''but''by''c''came''can''cannot''cant''can\'t''caption''cause''causes''certain''certainly''changes''clearly''c\'mon''co''co.''com''come''comes''concerning''consequently''consider''considering''contain''containing''contains''corresponding''could''couldn\'t''course''c\'s''currently''d''dare''daren\'t''definitely''described''despite''did''didn\'t''different''directly''do''does''doesn\'t''doing''done''don\'t''down''downwards''during''e''each''edu''eg''eight''eighty''either''else''elsewhere''end''ending''enough''entirely''especially''et''etc''even''ever''evermore''every''everybody''everyone''everything''everywhere''ex''exactly''example''except''f''fairly''far''farther''few''fewer''fifth''first''five''followed''following''follows''for''forever''former''formerly''forth''forward''found''four''from''further''furthermore''g''get''gets''getting''given''gives''go''goes''going''gone''got''gotten''greetings''h''had''hadn\'t''half''happens''hardly''has''hasn\'t''have''haven\'t''having''he''he\'d''he\'ll''hello''help''hence''her''here''hereafter''hereby''herein''here\'s''hereupon''hers''herself''he\'s''hi''him''himself''his''hither''hopefully''how''howbeit''however''hundred''i''i\'d''ie''if''ignored''i\'ll''i\'m''immediate''in''inasmuch''inc''inc.''indeed''indicate''indicated''indicates''inner''inside''insofar''instead''into''inward''is''isn\'t''it''it\'d''it\'ll''its''it\'s''itself''i\'ve''j''just''k''keep''keeps''kept''know''known''knows''l''last''lately''later''latter''latterly''least''less''lest''let''let\'s''like''liked''likely''likewise''little''look''looking''looks''low''lower''ltd''m''made''mainly''make''makes''many''may''maybe''mayn\'t''me''mean''meantime''meanwhile''merely''might''mightn\'t''mine''minus''miss''more''moreover''most''mostly''mr''mrs''much''must''mustn\'t''my''myself''n''name''namely''nd''near''nearly''necessary''need''needn\'t''needs''neither''never''neverf''neverless''nevertheless''new''next''nine''ninety''no''nobody''non''none''nonetheless''noone''no-one''nor''normally''not''nothing''notwithstanding''novel''now''nowhere''o''obviously''of''off''often''oh''ok''okay''old''on''once''one''ones''one\'s''only''onto''opposite''or''other''others''otherwise''ought''oughtn\'t''our''ours''ourselves''out''outside''over''overall''own''p''particular''particularly''past''per''perhaps''placed''please''plus''possible''presumably''probably''provided''provides''q''que''quite''qv''r''rather''rd''re''really''reasonably''recent''recently''regarding''regardless''regards''relatively''respectively''right''round''s''said''same''saw''say''saying''says''second''secondly''see''seeing''seem''seemed''seeming''seems''seen''self''selves''sensible''sent''serious''seriously''seven''several''shall''shan\'t''she''she\'d''she\'ll''she\'s''should''shouldn\'t''since''six''so''some''somebody''someday''somehow''someone''something''sometime''sometimes''somewhat''somewhere''soon''sorry''specified''specify''specifying''still''sub''such''sup''sure''t''take''taken''taking''tell''tends''th''than''thank''thanks''thanx''that''that\'ll''thats''that\'s''that\'ve''the''their''theirs''them''themselves''then''thence''there''thereafter''thereby''there\'d''therefore''therein''there\'ll''there\'re''theres''there\'s''thereupon''there\'ve''these''they''they\'d''they\'ll''they\'re''they\'ve''thing''things''think''third''thirty''this''thorough''thoroughly''those''though''three''through''throughout''thru''thus''till''to''together''too''took''toward''towards''tried''tries''truly''try''trying''t\'s''twice''two''u''un''under''underneath''undoing''unfortunately''unless''unlike''unlikely''until''unto''up''upon''upwards''us''use''used''useful''uses''using''usually''v''value''various''versus''very''via''viz''vs''w''want''wants''was''wasn\'t''way''we''we\'d''welcome''well''we\'ll''went''were''we\'re''weren\'t''we\'ve''what''whatever''what\'ll''what\'s''what\'ve''when''whence''whenever''where''whereafter''whereas''whereby''wherein''where\'s''whereupon''wherever''whether''which''whichever''while''whilst''whither''who''who\'d''whoever''whole''who\'ll''whom''whomever''who\'s''whose''why''will''willing''wish''with''within''without''wonder''won\'t''would''wouldn\'t''x''y''yes''yet''you''you\'d''you\'ll''your''you\'re''yours''yourself''yourselves''you\'ve''z''zero');
return 
$stopwords;
}
« Last Edit: October 05, 2007, 09:52:39 AM by herbacious » Logged
perkiset
Olde World Hacker
Administrator
Lifer
*****
Offline Offline

Posts: 10096



View Profile
« Reply #1 on: October 05, 2007, 10:12:25 AM »

Clarification - Herbacious posted this with the question about how to make his keyword filtering etc more efficient and powerful. I invited him to post here for more thorough PHP discussion on it.

/p
Logged

It is now believed, that after having lived in one compound with 3 wives and never leaving the house for 5 years, Bin Laden called the U.S. Navy Seals himself.
perkiset
Olde World Hacker
Administrator
Lifer
*****
Offline Offline

Posts: 10096



View Profile
« Reply #2 on: October 05, 2007, 10:19:45 AM »

herbs - would you please post an example of *exactly* what you'd like to see on output ... withouth recreating your linkify table et al it'd just make deciding what to do a lot easier...
Logged

It is now believed, that after having lived in one compound with 3 wives and never leaving the house for 5 years, Bin Laden called the U.S. Navy Seals himself.
Pages: [1]
  Print  
 
Jump to:  

Perkiset's Place Home   Best of The Cache   phpMyIDE: MySQL Stored Procedures, Functions & Triggers
Politics @ Perkiset's   Pinkhat's Perspective   
cache
mart
coder
programmers
ajax
php
javascript
Powered by MySQL Powered by PHP Powered by SMF 1.1.2 | SMF © 2006-2007, Simple Machines LLC
Seo4Smf v0.2 © Webmaster's Talks


Valid XHTML 1.0! Valid CSS!