• NEW! LOWEST RATES EVER -- SUPPORT THE SHOW AND ENJOY THE VERY BEST PREMIUM PARACAST EXPERIENCE! Welcome to The Paracast+, eight years young! For a low subscription fee, you can download the ad-free version of The Paracast and the exclusive, member-only, After The Paracast bonus podcast, featuring color commentary, exclusive interviews, the continuation of interviews that began on the main episode of The Paracast. We also offer lifetime memberships! Flash! Take advantage of our lowest rates ever! Act now! It's easier than ever to susbcribe! You can sign up right here!

    Subscribe to The Paracast Newsletter!

Request For Feedback

Is it time for a new order of data management in the paranormal field?

  • No

    Votes: 0 0.0%
  • You're way off the mark

    Votes: 0 0.0%
  • I don't care

    Votes: 0 0.0%

  • Total voters
    5
  • Poll closed .
So I guess several different departments don't want to discuss this... No reply. Oh well! Judging from the odd black sedan that was ever present this holiday weekend I know they still have the love... What a boring detail for some poor AFP team that must have been.

Moving on back to the real world.....

I am troubled... Maybe a moment of doubt. Is it really worthwhile constructing this taxonomy or even putting it in motion as a service?
 
Go for it. Someone else will eventually get round to it, why not let you be the person to initiate this worthy cause?
Go for it.:)
 
I've attached an initial UML diagram to get you started from the initial post you made in this thread.

As you'll see I've left a lot of it generic so it can be expanded as needs be. Some explanations to fill in the (massive) gaps.

Incident allows you to record not only UFO related, but other paranormal incident so you're not fixed craft sightings themselves i.e. you can record abductions as well as sightings. You could record things such as ghosts as well.

Trust ranking is a placeholder for a reputation system that can be used against incidents, participants in an incident, a contributor or a document. You could also extend this to create a reputation ranking using this and an array of contributors that have 'trusted' the record under question to derive a histogram against all records.

Incident_craft could contain multiple Physical_characteristics so searching over very general characteristics down to several that all need to apply is possible. Physical_characteristics can also contain entity characteristics e.g. big black eyes, grey skin etc.. so you could find incidents where there was a "metallic" "egg" craft with "furry" "feline" entities as an example.

Incident_type and Entity_type have parent Objects which is just a reference to the same type of Object but allows you to build nested types and sub-types, I;m thinking that you could populate the Entity type with common classification tables. The Incident_type could start general with say UFO, Abduction, Ghost, Vampire, MIB etc and then you sub-type from there. I could split this so that parent is removed and you have a joining object that allows a sub-type to belong to numerous parent types?

Participant and Contributor can/need be extended much more but it's a start. Contributor is synonymous with 'User' but I think it's a friendlier term - any contribution is welcome! :)

Document can cover anything from a multi-page pdf to a back of the napkin drawing, so long as it can be scanned and uploaded we can track it.

We could look at adding 'Tags', but you can kiss a certain amount on integrity goodbye.

As this is UML, I haven't added any of the method calls for the classes yet, or any of the relations (you can probably already see the obvious ones such as multiple participants to an incident and multiple crafts to an incident as two examples). I've just documented the initial properties and we can throw these out, refactor them into new classes or rename them. I've tried to keep them from slipping into a straight class -> database table mapping, but it's pretty close atm.

Cheers,
bb
 

Attachments

  • paracast-uml-v0-1.png
    paracast-uml-v0-1.png
    68.9 KB · Views: 11
Ha! The first non-blank shot is fired! Good on ya BB! Everyone stay tuned... I think our benefactors may be adding something to the conversation soon too.
 
you'd need a much better defined business case, and most likely have cut some code for a prototype before you'd get any serious investment. Besides, who says you can't do things on a shoestring :)

The main thing we need at the moment is people's time(skills), thinking and critical feedback/input.

bb
 
I created an excel file (created via openoffice so it may be basic in appearance). I've started populating it with values for the different taxonomies (types) that might be used. Where can I upload it to? This forum board doesn't accept .xls files - it's geared only for image extensions.

thanks,
bb
 
BB/All,

As the management of documents and lists for this effort has been on my mind for a while I have started a build of a 'project site' that will provide a systematic means of capturing and evaluation contributions including:

  • Feature Requests
  • Glossary
  • Data Model (Entities and Attributes now, Methods and Transports later)
  • File Material
I had started this as a Google site, however after some discussion with our esteemed benefactor, Gene has agreed to host said site beneath the Paracast. It is now under construction and expect to have it viable for use in the next 2 weeks max and launch... provided Gene is happy with the result.
Stand by.
 
WOW! Great thread. I am sorry it took me so long to see it. Anyway, I have been down this road many times over the years though never from a "For Profit" standpoint. I would be MORE than happy to trudge down it again. Count me in on helping with any initiative. Gene, if the traffic or DB data size looks daunting, let me know and I will help host it. by the way, i bough the domain (ufo-database.com) a few years ago. If you like you can use that domain as I am not using it.
 
We've already set up a site for development, and your assistance will be welcomed, I'm sure. Our server storage is 1TB, and we've only used a small part of that. We have no bandwidth limits in any practical sense.
 
thinking aloud, but perhaps we might be better off with something like Trac so you get the project requirements going under the wiki, tickets form the requests and you can allocate them into milestones / version so you can track and plan your burnrate. Integrates nicely with various CVS's.

Using Wordpress would be good as the promotional and blogging tool cos that's what it does best.

Using a wiki can be a truly open process which anyone can share in.

bb
 
WOW! Great thread. I am sorry it took me so long to see it. Anyway, I have been down this road many times over the years though never from a "For Profit" standpoint. I would be MORE than happy to trudge down it again. Count me in on helping with any initiative. Gene, if the traffic or DB data size looks daunting, let me know and I will help host it. by the way, i bough the domain (ufo-database.com) a few years ago. If you like you can use that domain as I am not using it.

so you've been down this road many times, I'm curious and don't like making wasted journey's - what are the key failure points you've encountered then? What are the things we should do that shortcut some of the work? Do you have zee data and is there any reason(s) why it would not be usable? What are the skills you can help with?

thanks,
bb
 
RE the Wordpress... at this stage I beg patience, however the site will cater for item tracking (Features, Entities, Attributes, Glossary Terms). Using the Type+Views plugins there is virtually no content type we can't model, and we can certainly attach a wiki as well, which I support. I realise there are free tools out there that can do a much better job of wrapping project info, however I'm keen to keep all the content as close together as possible. This benefits our hosts and promotes a tighter critical mass with fewer hops for participants. BB, your last entry reminds me that having a Risk/Opportunity register might not be a bad inclusion.
 
Like I said before on this topic, it's still garbage in = garbage out. So the answer isn't another web-crawler to database to screen approach. We need good solid cases that have been screened by ufologists, researchers and competent enthusiasts. I've already started doing something along these lines on the USI website, but the problem is that it takes a lot of time and effort, more than I alone have time for. Our search database is deceptively simple. For example, although it seems basic by comparison to a relational database, it is scalable using JavaScript and in theory most of the same functions are possible to the extent that any researcher would need them for any practical purpose. Even now, it is working reliably and will return results based on specific keywords. Visit the USI website and type something like UFO into the Search feature. You should get back around 59 results, each linked to a corresponding entry or article, all laid out in a user friendly interface online. This is our own proprietary search feature. It can be used offline without any server, extra plugins or software to install. The whole thing can be put on a flash drive or CD and run from there. We also have the various sighting classification models outlined on the site for reference, plus a unique multi-star overall confidence rating system. Plus we developed a mnemonic rating (M) category in addition to the Vallee SVP system for the purpose of addressing sightings involving altered states of consciousness, something that researchers in the Early Modern Era either avoided or didn't run across as frequently. So IMHO we don't really need a complex relational database application right now as much as we need quality content ... volunteers to collect, evaluate, and input the data ... if anyone is actually interested in doing any volunteering, please get in touch here or via the USI website.
 
I disagree with some of your points.

It isn't another web-crawler to database to screen approach - that would end up as yet another narrow minded view of what the data is and shaping it to fit the author's view of the world.

Garbage in can be useful. For example, hoax sightings can act as teaching aid of what to look for that's 'dodgy' about a case, you can mine them for memes or common traits which can then help in making initial classifications of new entries. Garbage in is also a subjective decision - for example, you might get a case in which on face value has no intrinsic value, indeed it may not have any value for years until other cases are found/ recorded with similar traits, so by throwing out ones which offer no interest to you, you could be throwing out a potential nugget that after more time/ data is afforded for analysis proves highly valuable.

I do agree it takes a lot of time and effort, but that is why I am interested in this project, collectively we can use our time and efforts for the benefit of others and put in what we can and take out what we want. One of the the intentions being that you can put what data you want in, and take out data is several formats to do what you want with it. So, you could export the data as say Javascript arrays and then use them in your site so you'd have some cases under the search. Somebody else may take some data and create a mobile app focused around that data, someone may take that data and look at abduction cases and mingle it with their own data to provide more context. Putting it in a relational database gives flexibility to then query the data, or facets of it in lots of different ways according to the individual's needs, not by being forced into some convoluted search form that only exposes some dataset that the administrator deems worthy. Getting all the data and segmenting it how you want is one of my goals, whether that be by plotting some pins on a map having uploaded data into Google, using a pivot table in Excel, using a report in an Access database, building your own mobile app to see what has been sighted around your location or just typing CTRL+ F in a text editor and finding all occurrences of 'ufo' to be curious about what's in there.

If you have expertise on nomenclature and categorisation of 'things' then why not help us out when the Wordpress site is available, the whole point is that things will be put for advisement, consideration and to be edited. If you don't like the way something was done, then you could download the data and mangle it yourself on your own machine and explore it - at least the data would be used and would be subject to scrutiny rather than lying around in databases that are accessed by only one person for their own personal study - think about what happens when that person dies, or loses interest... or their machine drive goes AWOL (you lost a machine in the 1990's and the member database due to a crash)... all that data/information/effort is lost. It does not have to be that way.

Few things with your site. Try typing 'u' in the search. Firefox and Chrome just grind for ages, IE probably does as well. Type 'ufo cases' into the search - there are no actual cases in the 59 results returned - just articles about various personalities within ufology and definition of terms, so I'm unsure if you're more a dictionary/ reference site or a case provider as well?

Using frames tends to be bad for your site both in terms of SEO and is also a generally accepted UI bad practice. The search seems to indicate there are 144 records in total, but it only ever seems to offer 59 max?

With a bit of reworking (using Javascript) you could make the site work faster offline (at the moment I'm assuming that you mean you have to manually save the pages discretely off to the local hard-disk) and mobify it for other device use(iphone/androids etc). If you've got the lat/longs of any cases, or indeed if you geo-located the information-rich pages you do have in there, then you could create a "Layar" layer and people could again use your site in a more location aware context.

I hope you will take this reply as means to explain why this effort is beneficial to you and to a much wider community and not as a attack on what you've done, which is good - heck, you've got something out there :)

Ufology as a whole needs a kick up the arse and needs to become more open, it'll help expose the frauds quicker, get data out there quicker for people to assess, theorize and postulate on and you might even find some answers quicker. It also needs more young people to get involved and providing the information in bite-sized chunks or a more engaging way than a book or a long web page might be the difference between tldr; and someone posting questions back somewhere until they're hooked enough to read the book or the long webpage :)

Constructive criticism is always welcomed. I'd happily make further suggestions for your site if you thought it of benefit, you don't have to do anything with it, but sometimes it's good to have a fresh pair of eyes?

bb
 
so you've been down this road many times, I'm curious and don't like making wasted journey's - what are the key failure points you've encountered then? What are the things we should do that shortcut some of the work? Do you have zee data and is there any reason(s) why it would not be usable? What are the skills you can help with?

thanks,
bb

Just a quick...ish note. This topic can really get me going so I could easily write a book.

As for me and my abilites, I own a software company, an IT/IS consulting company, and am a partner in a few other web based businesses. I also co-own a private data center and disaster recovery facility. I have developed and architected enterprise software solutions for over 15 years and have extensive data mining expertise. Oh, and I grill a mean steak too! :)

Like everything associated with this field and phenomenon, trying to consolidate UFO data is a story laden with drama, dripping with frustration, and burdened by short sighted and "me"centric ego. It is gonna be a hard row to hoe.

Pitfalls and Problems
1. Access to Data. MUFON is the widest and largest UFO related data collection and consolidation entity in the world not run by a government or reclusive billionaire. Probably 85%+ of the worlds reports end up there. Personally I think this is a conservative estimate. This collection runs for over 40 years. A few years back MUFON sold their entire data collection effort to a notorious figure in Las Vegas and the entire collection was boxed up and sent there. The contract between MUFON and Bigelow was leaked and is displayed somewhere in the forum archives in its entirety. Its a dry but interesting read. One interesting part is that the Bigelow group got everything and only had to return something like 3 or 4 pieces of data to MUFON. (Bob Bigelow Bought MUFON ?? | The Paracast Community Forums)

It is my recollection that part of this contract was that MUFON could not grant unrestricted access to the data it retained. But I have not read it in years and cant remember all the details. Instead MUFON created an extremely worthless database search engine that gives very little if any relevant research data. But, it does give MUFON the ability to represent themselves as having a degree of transparency of data. To be clear, they do not and are not at all interested in hearing about data search patterns, cross correlative data analysis, historical data reconstruction, or any number of other ways the data collected could be used to garner information.

Of the remaining 15% of collections, most are guarded by people or entities that do not want to share their data. Most of my pleas were either unanswered or I was given a "not at this time" response. But, in fairness, I was one man. I feel now and have always felt that a group would get more attention and perhaps elicit a different reaction.

2. Deaf Ears. I have personally plead the case to MUFON about truely opening their database to qualified researchers. I suggested things like creating a qualification or certification program, creating a MUFON profit center by allowing researchers to pay for access, Requiring researchers to sign very restrictive NDA's, and many more ideas. But no. ALL of my suggestions were ignored or shot down and I kept being referred to the same bullshit database search engine you get access to when paying for a MUFON membership. Frustrating.

3. Data Integrity/Consistency. How do you ensure that the data you are collecting is reliable and unaltered. I have seen the same incident reported in 3 different ways from 3 different sources. If you have been looking into this subject with any depth you have seen the same thing. The problem is that some of these stories are so freakishly divergent they almost need to two or three different occurrences. So much of this phenomenon seems to be perception. When you have 3+ stories your going to get 3+ different perceptions of the events. How do you consolidate these and still maintain consistency, cohesion, and integrity? It becomes a really difficult job very quickly. All of this only compounds when your data provider is sketchy.

There are more negatives, but these are the biggest issues. NOBODY wants to give up their data. Sure they will give you small tid bits or point you to their book. But if you want to have a database that is useful for research it must include details that are more often irrelevant to a story. For instance, in a book I might say, "the object was seen on the ground in Macon County". For a book that might suffice. But a researcher might want to cross correlate data points with other databases or services. I might need to know exactly where in Macon County so that I can lookup common vegetation, topography, soil composition, population dispersal, etc.

Even congenial researchers/authors that preach openness are not willing to provide their data. Some will angrily tell you to go do the research yourself. Others will promise and never quite "make the time" to get you what you ask for. I have even been told that I could buy all the research a certain guy had for $2000. I declined.

BUT, and I want to stress this, lets not let history deter from trying again. Its frustrating and sometimes you have to take a step back for a while but the pay off would be too cool to "go quietly into the night" on the idea. I am still willing to help and it looks like the forum is attracting more of the technical crowd, much to my pleasure, so why not give it another try.
 
Back
Top