EDemocracy U.S.

Tech Committee


Ready to volunteer? 

Drop us a note.

eWeb - Usable e-mail list archives on the web, future equitable discussion platform


eWeb Requirements

Steve Peterson and Steve Clift sat down and started work on a detailed technical requirements document.  Below are Steve Clift's suggested edits.  Requirements need to be broken down into eWeb Phase 1, eWeb 2 Phase 2, and suggestions to those developing Mailman or for Mailman patches.

Last updated November 18, 2002 by Steve Peterson

#   Short description   Notes 
1 Read email from POP mailbox with user configurable polling schedule 

2 Put raw messages in a relational database Do we want the messages themselves in a database or in text files? 

3 View messages on web -- one message per page 

4 View messages on web -- one thread per page 

5 View message index by date 

6 View message index by subject 

7 View message index by author 

8 Multiple message stores per server 

9 Basic keyword search through messages Lucene? 

10 Cross list search 

11 Advanced search Search over range of dates. 

12 User profiles It's what all of the user's preferences are hooked to 

13 Set retention period for messages in days 

14 Set administrator email / name / password for feedback on list Should be per-list option 

15 More than one adminstrator per list? Not sure it's needed 

15 Administrator of more than one list 

16 Link from pages to the lists's "site" Should be both a label and the underlying URL. Offer an option to put the "originating message" on the URL? Could get the same thing by analyzing the referrer 

17 Scalability Would like to be able to handle many lists with lots-o-messages 

18 Show dates in user's time zone 

19 Select time zone to show dates in when there is not a preference to override it 

20 Custom HTML in various places Need to ID where this would be useful. wixList has header, footer, list info 

21 Process HTML in messages Strip it out completely or try to keep some of it around? 

22 Quoted-printable processing 

23 Logging 

24 Attachments Keep them? Ditch them? 

25 Maximum message base size 

26 Maximum size per-message 

27 Generate RSS feed from message base Do this on-request, caching it. Have a settable frequency for regeneration. Keep track of whether messages arrive to make sure that you only regen when needed. 

Steve Clift: This site "collapses" the threads, obviously this is a bit messy 
because the message/thread IDs (whatever they are) get changed by 
I'd syndicate the collapsed threads to have higher variety. 

Here is some info: 
Example: http://www.mail-archive.com/do-wire@t.../maillist.rdf 
(save as text on IE, netscape loads as text file) 

28 Save a search with a profile & get email when a new message arrives that matches the search term 

29 Must provide a view into the message base that is 
indexable by external search engines 

30 Recent messages view 

31 CSS based formatting to allow for stylistic flexibility; per-list CSS? 

32 Get rid of quoting when possible 

33 Statistics and graphs This needs to be expanded 

34 Hot topics 

35 List posting 

36 Email poster 

37 Rethread messages Thread IDs don't always work, see if there are heuristic approaches to doing this better 

38 Share email with friend 

39 More fully featured digests, maybe full text, topic only, excerpted 

49 Placeholder for message display stylistic requirements www.phorum.org, geocrawler.com, yahoo groups, communitybuilder, 






50 Profile viewer A la 

51 Any special list software integrations? 

52 SMTP server Cut the POP3 mailbox out of the loop, subscribe directly. Posts appear faster in list. 

53 Mailman integration 

Steve Clift's suggested edits/additions

Here are my edit/additions that haven't made it into that draft:

In requirements sheet (above):

5-7 View Message notes:
Range variable - week, month, year
Popularity - web archive hits, "good post" eclicks
Thread posts in linear fashion w/extra >'s cut out, grey italics
previous text

11 Advanced search - map kazoom?, related posts (auto links next to
post displays?)

default views, css skins

> lines
Replies = R
Excitement = Time Between Replies
Momentum = R/Days

Summarizer - like journalism news site, google news?

Link to related Google search - based on subject?, descriptive
keywords with list geography term

16 - link from pages - grab index page title, cache page/file -
w/size limits

18 - show date in user time zone - no - default display the elists
time zone

23 - logging - create simple link out urls (replace links in sent e-
mail with urls unique to list - create AXS-like link out stats - Most
Followed Post Links - test links for spaces, if error combine lines,
erase spaces

24 - default - ditch attachments, remove HTML

27 - RSS
Put on to your site code option
- Recent Posts - Today
Subject line four words ...
Subject line four words 2 ...
- Top Topics This Week
Subject line four words ...
(based on reply volume)

Additional items:

Topic/Word Trends

Top Cited URLs

Eclicks - things you can click in the body of e-mail to express a

Forward to friend - mail/thread eclick -> from site send URL and
Subject or Full Text

Recommend Forum to Friend

Color Posts in archive based on ranking??

Heuristics - flame/spam

All e-mail poll feature w/web and e-mail results

Cross-posting analysis - show where else a post has been sent if
archiving multiple lists

Backup features

Post from/by location/zip code/city
Map posts/posters
Posts per capita - hmmm use census data

Posting averages - # recepients
- display historical reach - e-mail XXX, web views XX, eretreived X

Altavista babelfish integration/other translation link

Spam harvest defense

Reply via web to list/author
- within time date/range
- per list option
- login required/not required - moderation option

URL checker - deactivate dead out links

All e-mail settings - send commands one at a time or [X] text files
returns with e-mail "ok" confirm

What's New - automatic stuff based on archive activity

Altavist Babelfish/other translation tool link integration

Spam harvest defense

Reply via web to list/author/manager
- per list option
- logged in/not logged in option
- moderation option

Error links deactive linkouts from archive

URLs to any output shorter than 70 characters and intuitive

URLs to search queries linkable like Google

Cross site - What' hot w/collapsed threads for public lists

Fresh Faces - ID first/first five posts from any address

Banner ad rotation - perhaps cross list promotion service window

Member directory
Or /dir/
Login link to update/add optional info to member page - perhaps allow
e-verification w/o password? Possible listings: Member level, donor
status? Status indicator - warnings, suspended First post, recent
post, Total number of posts per list

List Enhancements

- allow full subscribing/feature trweaking via e-mail
- Reply "OK" should work

Allow # posts per member per X hours/one day period to be limited
Allow total # posts per extended period of days - result two a day,
but only seven total in a week

Excess quoting return to poster/strip option for quoting at end of

Insert archive URL in the top
Insert archive URL for archived replies in bottom of each post

Insert members page URL

Insert unique to post and receipent web/e-mail ranking link in
message footer:
+1 substance (allow variable terms)
-1 style
up to six ranking items
Anonymous yes/no option per list
Alternative Rate url with web based rating options

Archive Side bar
- related posts based perhaps on frequent unique keyword from subject
and body? - Join - encouraged everywhere

Attachment options - scan virsus, keep in archive, strip

Legal liability - ability to universally remove anyone persons posts -

"Freenet" anonymous protection options for non-e-democracy uses

Auto e-mail from list to archive
- weekly subject highlights
- stats, hot threads, viewed posts on the web
- # posters, poster diversity stats
- auto-highlight option from other lists

Momentum alerts
Bursts - speed of replies X level
Stars - threads with extended length
Leaders - Posters starting threads with most bursts, stars, total
thread length, poster diversity

Header preservation tools - reconnecting message IDs

Multiple/same IP tracking/blocking
(can't recall what I mean here)

Robot/spambot issues

List nomination database - allow people to select topics for future
list, reserve their spot

Get comments from do-code - Landfield/Hutsinger on "Objects" mini-
aps comments

Cascading style sheets, ssi options

Skins/templates - define sections of archive pages - NO FRAMES EVER

All e-mail request processing - design for low bandwidth options

Login to post via web - uniform password with all Mailman lists

All/multiple public lists by date
Date by subject

Forward thread url

Content analysis options - hooks to social science content analysis
tools, perhaps allow them to be used to present views of archive

Dmoz/Zeal appropriate section nominator


All web based admin, no shell access required

News summary features - academic sites, Google News

Notes from meeting what kind of behavior changes are we looking for -
means - measures

E-mail material
- haven't post in a while reminder
- boy you post a lot
- surveys
- etc.