@tobez: "Hi folks,
I would like to point <url> to you, and hopefully
get you to check it out and make your comments and suggestions.
Below is the contents of <url>
Port-tags is like del.icio.us [<url> for FreeBSD ports
collection.
The idea is to make ports classification easier and more convenient.
Instead of using predefined and limited set of port categories,
port-tags uses short one-word descriptions called tags. A port can have
an arbitrary number of tags associated with it. One can use the web
interface (and maybe a command-line interface in the future) to view
only the ports that have particular tags associated with them. This
process is very efficient in narrowing down the number of sought ports.
Debian has something similar called Debtags
[<url> The main difference of Debtags and
port-tags is not the absense or presense of particular frontends.
Rather, Debtags takes potentially more fruitful approach of getting the
community to define the tags, while for now, port-tags uses an
algorithmic approach to deduce the tags from the existing categories and
from ports' COMMENT field. Let's see how this works out and extend it to
a community-based project at a later stage.
The idea of port-tags came up in a conversation I had with Autrijus Tang
a couple of weeks ago. Since then, I discussed it with various people
(most notably with Lars Thegler, Erwin Lansing, and Florent Thoumie) and
finally implemented it.
In its current form, the service is not updated live. Instead, as it
depends on the INDEX file, and the index file gets updated once a day,
so does the port-tags database.
If it looks too similar to del.icio.us, it is because for this early
alpha version I just stole the stylesheets from there.
If you have any comments and suggestions with regard to port-tags, I
would like to hear about it. Is it useful? How can we make it more
useful? What are the shortcomings? Any show-stoppers which prevent you
from using it? 
Thank you for your attention,
\Anton.
An undefined problem has an infinite number of solutions.
 Robert A. Humphrey"
    @ringworm01: "<quote>
I'd like to see a copy of the tags data base. Providing it in a similar maner 
as the INDEX file might come in handly.
-Mike"
    @tobez: "<quote>
<url> - it's an sqlite3 database
currently.
I will publish the source code in a couple of days time, too.
But speaking about INDEX, it is not inconceivable that we could extend
its format to include tags as the last |-separated field - if the idea
takes off.
Cheers,
\Anton.
An undefined problem has an infinite number of solutions.
 Robert A. Humphrey"
    @ringworm01: "<quote>
Personal preference is a seperate data base, it will slow down generation of
the INDEX anyways if you combine them and will be more efficient if set up 
correctly.  A flat text file IMO is ideal.  A quick glance at your is it 
contains too much info. Make it simple and I think more programmers/scripters 
will make use of it.
A nice easy layout like:
tag1	| port category | port
tag1	| port category | port
tag1	| port category | port
tag2	| port category | port
tag2	| port category | port
tag2	| port category | port
This all that is really needed, from that pkg-desc or comments may be easily
retreived/searched because your database is provideing a quick index to them, 
then all you have to do is concentrate on making the tagging portion smart.
My $0.02
-Mike"
    @tobez: "<quote>
This is not very difficult to arrange.  :-)
  <url>
Updated with the same frequency as the database itself, that is, for now
daily.
\Anton.
An undefined problem has an infinite number of solutions.
 Robert A. Humphrey"
    @ringworm01: "<quote>
Now that is starting to look handy, from here how hard is:
from:
opengl|devel|esdl
opengl|devel|mingw-opengl-headers
opengl|emulators|linux-pete-mesagpu
opengl|emulators|linux-pete-xgl2gpu
opengl|games|BillardGL
opengl|games|airrox
opengl|games|briquolo
to 
Things that use opengll|Things that use java|devel|esdl
Things that use opengl|Things that use java|devel|mingw-opengl-headers
Things that use opengl|Things that use java|emulators|linux-pete-mesagpu
Things that use opengl|Things that use java|emulators|linux-pete-xgl2gpu
Things that use opengll|Things that don't use java|games|BillardGLThings that 
use Things that use opengl|Things don't that use java|games|airrox
Things that use opengl|Things that don't use java|games|briquolo
People would have to agree to a reasonable depth, I assume your program wants
to subcategorize a few tags anyways so at least three levels of tags?  If you
do this I'll put something rough together that for now just prints a listing, 
later something more useful will come to mind.  Will you be keeping this 
database updated on your site for awhile or is this just a short term 
experiment?
-Mike"
    @tobez: "<quote>
Now, it is not particularly difficult to do that, but things WILL
explode in size quite a bit, because, if we just take for example
devel/esdl [where did you get the idea it uses java, anyway?  ;-)], we
will quickly come to
access|devel|esdl
devel|access|esdl
access|library|esdl
devel|library|esdl
library|access|esdl
library|access|esdl
access|opengl|esdl
devel|opengl|esdl
library|opengl|esdl
opengl|access|esdl
opengl|devel|esdl
opengl|library|esdl
access|dsl|esdl
devel|dsl|esdl
library|dsl|esdl
opengl|dsl|esdl
dsl|access|esdl
dsl|devel|esdl
dsl|library|esdl
dsl|opengl|esdl
- only for positives.  Adding negatives here,
we'll have to list every tag which is combined with "devel" and is not
"access" or "library" or "opengl" or "dsl", and say
devel|! abstract|esdl
devel|! column|esdl
So even for two levels, what you ask pretty much does not make a lot of
sense, provided I understood your idea (and please explain in to me one
more time if I did not)!
Or to rephrase: I use an SQL database for a reason in the web front-end.
The database is going to live there for a while, yes.
\Anton.
An undefined problem has an infinite number of solutions.
 Robert A. Humphrey"
    @ringworm01: "<quote>
One common gripe about the ports system is the two level directory structure 
is too flat, and certain folk resist rather vigorously adding sub categories. 
The sugestion put forth was to use tags as a multilevel directory structure
and using the actual structure only as place holders, so your program
could do something like this:
we all ready have:
sysutils
editors
games
like to have
tags						dirs
opengl|sysutils|file related	-=> /sysutils/some file program
opengl|sysutils|file related	-=> /misc/some other really cool file program
opengl|editors|java based	-=>/java/some java editor 
opengl|editors|notjava based	-=>/editors/some editor based on opengl
openglgame|lame			-=>/games
openglgame|not lame		-=> games
The idea is to decouple the actual ports directory structure, and come up with 
something more useful without having to change the actual ports structure.  
If your tags are only one level deep, we aren't much better off than before.  
Maybe for starters use the categories as they are for the
first level then see what a second level would look like with your program?
-Mike"
    @tobez: "[moving back to the mailing list since this might be of interest]
<quote>
Currently it basically takes the existing categories a port is in, plus
the words which constitute the port's COMMENT.  Then it applies a number
of heuristics, most significantly stemming and filtering out the common
"stopwords" like "a", "the", and so on.  And then there is of course a
cutoff for those resulting tags which are too rare (otherwise the number
of resulting tags in the tagcloud would explode;  it is already pretty
bad as it is, with 815 tags).
Since mail/dovecot does not mention "maildir" in it's COMMENT, no tag
for it.  I was thinking about parsing the pkg-descr file as well, but I
was afraid it will explode the number of tags even more.  Possibly I was
mistaken, it would be good to experiment with this approach.
Failing taking into account pkg-descr, one can clearly see why a "social
collaboration" mode could produce higher quality results, since at least
one person (you) would make sure that mail/dovecot is tagged with
"maildir".  :-)
\Anton.
An undefined problem has an infinite number of solutions.
 Robert A. Humphrey"
    @ringworm01: "<quote>
OK, but if someone wants to browse the port tree to see what is there, how are 
they going to do it when the initial index is 64,000 lines long? 
-Mike"
    @tobez: "<quote>
No argument here, more or less.  We, however, seem to disagree about the
actual implementation.
Maybe we should not think about tags as categories and dispense with the
idea of a `level' altogether.
Tags, as opposed to multi-leveled categories, are equal to each other.
One needs to be very careful imposing a structure where there is not
one.
So instead of thinking in terms of tags as multi-level categories on
steroids, the idea is to be more operational:
"I want a _mail_ _client_ that supports _maildir_ and _imap_".  No
levels.
The actual technical implementation becomes much more sane, too. Think
"joins", as opposed to pre-populating huge text file with all [alright;
not all;  some;  who gets to define which ones?] possible combinations
of tags.
So, to re-iterate, the right operational mode for this is not "go there,
see what's in it", but "impose a restriction, see what's left".
Are we on the same wave?  :-)
\Anton.
An undefined problem has an infinite number of solutions.
 Robert A. Humphrey"
    @ale: "<quote>
I had a similar idea some days ago (well, only an initial and 
theoretical idea, no implementation). It's great to see I'm not alone. I 
like it very much. One thing we can add to enhance the tag 
classification is to add an optional PORTTAGS variable where we can 
define additional tags (not present in CATEGORY and COMMENT) for the 
port, so that "Maildir" could be for example associated with 
mail/dovecot. This will require additional work for maintainers, but I 
think it'll highly improve the searches.
Alex Dupre"
    @tobez: "<quote>
Nice to hear.
Oh yes, I quite agree.  I'll incorporate the support for PORTTAGS (maybe
it sould just be TAGS?) into the indexer.  The best part is that no
individual maintainer needs to be forced to use this mechanism, the
indexer will fall bacl to CATEGORY and COMMENT if no tags are defined.
So the change can be gradual.
\Anton.
An undefined problem has an infinite number of solutions.
 Robert A. Humphrey"
    @lth: "<quote>
I suspect most (if not all) tag addition could be done within COMMENT 
itself. For people browsing the portstree through other means, say, 
FreshPorts, improving COMMENT makes very good sense, IMHO.
/Lars"
    @ale: "<quote>
Writing good COMMENTs is surely a thing to do, but that variable is not 
meant to hold many keywords. For ports where we can include all the 
relevant tags into the 59-chars COMMENTs, no additional work is needed. 
If the keywords are many and don't fit all in the COMMENT field the port 
maintainer could insert them into the TAGS variable.
Alex Dupre"
    @past: "<quote>
This is great! I've been thinking along these lines for some time now, 
but never found the time to do something about it. I was considering 
using lucene to create the inverted index from port comments and 
descriptions, like you did and building a gui for searching through it. 
If you keep maintaining the web site, it would be a good idea to provide 
some sort of web-service protocol in order to be used from client 
applications. It could be something as simple as querying by adding the 
terms in the http path info and returning a document with a list of port 
names (newline-separated, comma-separated or whatever). Perhaps it will 
be easy to add this functionality to sysutils/bpm or sysutils/portbrowser.
Cheers,
Panagiotis"
    @tobez: "<quote>
Yes, some sort of API would be useful.
I just added the following:
If you add a m=pl ("mode = portlist") parameter to the URL,
it will return a newline separated list of matching ports:
	<url>
For performance reasons, I limit the number of returned ports to 200 by
default.
So that
	<url>
will return
	E(too many)[433]
instead of all 433 ports.  This can be overridden like this:
	<url>
The same goes for tag-less queries:
	<url>
will produce
	E(too many)[13598]
But if you feel really adventurous,
	<url>
will return all 13598 ports.  Please note that "n" parameter does not
specify how many ports to return, it specifies the upper limit only, so
if that is exceeded, "E(too many)" is returned.
I hope this is useful enough - and the good part is that we can always
extend it further if we have good ideas.  :-)
Cheers,
\Anton.
An undefined problem has an infinite number of solutions.
 Robert A. Humphrey"
    @parv: "in message <email_address>,
wrote Lars Thegler thusly...
<quote>
When i read COMMENT, i want to know the purpose of the port.
Currently, in most cases i have to read pkg-descr to get a better
understanding (as COMMENT is just filler).  (Yes, i know that only
thing that can improve the current situation would be to file PRs to
better define COMMENT.)
If keywords start filling COMMENT,  i think purpose of COMMENT will
fail.  Then again, rereading what i wrote in the above paragraph,
COMMENT will server a fruitful purpose finally by containing
keywords (albeit under a indirectly related variable name).
  - Parv"
    @tobez: "<quote>
I will support a TAGS field;  what I cannot decide at this moment, is
whether to ignore COMMENT and pkg-descr (support for which I will add
shortly) when TAGS is present, or use TAGS as an additional tag source
only.
\Anton.
An undefined problem has an infinite number of solutions.
 Robert A. Humphrey"
    @tobez: "<quote>
It occured to me from some IRC conversations and from observing the
access log, is that a number of people only try to click on a single tag
at a time (producing understandable "so what?" reaction).
The real advantage of the interface is that, once a particular tag is
chosen, the "Related tags" cloud appears, allowing one to limit the
subset of matching ports even further.
"devel" is useless.  "devel+mail" is much more useful.
Just to be clear on this point.
\Anton.
An undefined problem has an infinite number of solutions.
 Robert A. Humphrey"
    @wxs: "<quote>
I would think that if someone goes through the work to define a TAGS
field that it would be more accurate than what your getting from COMMENT
and will be getting from pkg-descr.  In order to weed out any
less-than-descriptive tags I would use only the TAGS field if it exists
in the makefile and if it does not then use the combination of keywords
from COMMENT and pkg-descr.
BTW, I really like this idea - it makes searching for ports much easier.
 WXS"
    @dan: "<quote>
Anton and I have discussed this off line and he's added some examples 
on how to use port tags.
In short, as I understand them, the feature is an iterative key word 
search.
You start with a list of ports and all the keywords (tags) for those 
ports. You pick a keyword, and the list of ports reduces to ports 
with just that keyword.  Then you repeat the process with an ever-
decreasing number of ports and keywords, until you have a list of 
ports that is reasonble to read through.
Dan Langille : <url>
BSDCan - The Technical BSD Conference - <url>"
{'source': 'USENET', 'other_information': '{"reply_count": 21, "forum_name": "lucky.freebsd.ports.20140828.mbox"}', 'file': 'data/output/usenet-lucky.freebsd.parquet'}