Read Post

Sat, 14 Oct 2023 06:16:02 -0700
Andy from private IP
/all
Technical explanation for the simplicity of this board

I was designing this site in early September, and I realized that all it would
take to match the original JDU is a very simple filesystem-based BBS without
even using a SQL database.  This drastically simplifies everything, but adds
other challenges.  One advantage is that the board is fully portable and I can
have a complete backup system very easily that continuously syncs with the
primary.  This eliminates the problem of SQL replication, which is otherwise
currently beyond the scope of my web application projects.  The general approach
is using plain text files to contain the posts and putting them in an actual
file hierarchy with command line tools to extract the relevant data, which is
tagged.  I believe such large community sites as Craigslist use this same method
to handle their global high-traffic systems, because otherwise SQL replication
and performance are the two primary challenges.

In terms of disadvantages, it's a lot harder to have an effective search box
without a proper search index as would be found in any SQL database.  However, I
think I've come up with a blend of performance, convenience, and accuracy by
using standard command line tools in conjunction with plain text files.  The
Linux tools "grep" and "sed" running in a simple PHP script are all I really
need.  Grep will search text files with extreme speed, and sed will seek to
particular lines in the file and perform operations on them.  Sed is how I
extract subject lines and metadata for posts, and grep is used to find the reply
tag to show the number of replies.  The cool part about the search box is that I
will be able to add regular expression search in the near future easily, so
people who know how to use PCRE can find exactly what they are looking for.  I'm
not sure Google is going to index all the posts, but I'll figure that out.

I think Admin on the original JD Underground took an approach similar to this
one, although he had a traditional login system and it was always unclear what
type of technology he was using on the backend.  Either way, this was a fun
challenge to start and it will be a continuously improved project.  Feedback and
feature requests are welcome.

Replies require login.

7 users registered
Sign in | Register | Post width: chars