Request For Feedback

[LoremIpsum]

Whilst listening to the show over the last months an idea has occurred to me. I don't know if it's a new idea, or even a good idea, or even workable given virtually no resources. I don't know if, even in full fruition, it would be of any value to anyone. However, I would like to share the idea and my reasoning for it here, and encourage feedback with the hope that it might just interest someone. Gene seems somewhat interested, but then he is a technologist as well.
A Little Background
Before we start the meal, let’s set the table and whet our appetite. Here are some background premises and positions:

1. The first thing to understand is that I offer this concept as a regular guy with some years of IT experience across data management and business analysis. I am a family man more concerned with feeding my kids than discovering the truth about UFOs. I'm not chasing fame, glory or profit for profit's sake, and I'm not interested in going head-to-head with dangerous people. However, I have been an enthusiastic listener, reader, conversationalist and occasional participant in subjects off the normal radar since I was a young boy, and I try to keep an ear turned to such matters as and when I can. If nothing else, they comprise material that is more entertaining and healthier for my brain than anything being force fed over mainstream media! (Can I get an Amen?)
2. Having been a consumer of paranormal and exo-political material for so long, I am frustrated that all we still have as a collective community is a soft cloud of hearsay, documents of various dubiousness, popular mythology and a healthy book publishing market. It's all very entertaining (and profitable for some), but isn't there more we could achieve here? In fact, is there already substantive evidence buried beneath politics, commercialism and counter intelligence that might be lifted up, strung together and made to tell a story? I am not so naive as to think anyone can just pop out a single ‘smoking gun' piece of evidence for any topic, or else it would have happened by now. But is there a way to use the tomes of material already out there to get a better picture? Is there a way to go about it that condenses the combined labour and learning curves of serious researchers into a format ready for mass analysis... ready for more rapid consumption and scientific scrutiny? If we could start to identify patterns and relationships in existing data that went beyond mathematical coincidence what would we start to learn? How would it begin to refocus research efforts? How would it change opinions held by the 'establishment'?
3. There is a ton of data! So much so, especially now that more official documentation is being released through the flood gates, that for any normal person traversing it in search of any gems or threads of truth it seems an insurmountable task. The data is scattered... collected in private databases, stored in authored volumes from backwater publishers, held still in the minds of witnesses and , I'm sure to some extent, protected in public and private sector vaults and facilities. And worse yet, most of this data is not indexed to any particular meta-data standard. We are left with keyword search and second/third hand verbal accounts. And frankly, though the internet is great it is not a controlled or standardised information source.
4. We live in an amazing (if not frightening) time with regards to available technology. In my work I deal daily with software and hardware that allows the organisation to manage and interrogate unimaginable scopes of data. At home, anyone can for a pittance setup a website, electronically manage their budget and CD collection, stream questionable video to their friends... etc. Open source software combined with the skills we use in our day jobs pose a formidable force. And then there's mobile technology! Hmmmm...

The Idea
Would it be possible to develop a community-driven, highly searchable, open-source, relational database to begin capturing old and new data related to off-radar topics (with an initial focus on sightings)?
How many enthusiasts and serious researchers would be keen to interrogate a database containing 50 years or more of sightings data by specific attribute? Would it useful to be able to visualise on a global map the concentrations of sightings? What if you could choose to also visualise this over a time sequence including vector of travel information? What if you could find clear correlations between sightings of strange phenomenon and events noted in specific government or corporate documents? Would these correlations better inform us as to which events seem associated with terrestrial authorities as opposed to other-worldly influences? Would the apparent vector paths and concentrations focus our exploration around specific geographic areas? Would we begin to see the annual migration pattern of the humble Sasquatch or clearly visualise activities in proximity to cave systems? Could we begin to filter the data via ranking according to a set of criteria established to weight the probable validity of an incident (e.g. volume of witnesses, physical trace, etc?)
In a situation where 'hard' evidence is seemingly impossible to come by for whatever reasons, we are left to rely on the field of ambient information with which to draw conclusions. We must create evidence from patterns.
Such a database would adhere to at least the following guidelines:

1. Principle: The taxonomy for meta-data would be extracted to the highest level and attempt to categorise entries across a range of useful attributes. Some examples for 'incidents': Geo-location, typing attributes (categories), temporal attributes (date/time), physical characteristics, positional disposition and vectors (motion, range, altitude, direction, etc.). The key here would be a common taxonomy across topics within a single grouping... meaning that Incidents would include sightings or interactions with craft, objects, phenomenon or biological entity of any kind. For Documents you would have a different set of meta-data, but effort would be made to create points of intersection.
2. Principle: The user interfaces must be designed for ease of use. A massive database structure on the backend will be useless if it is too difficult to get data in or out. And in the situation where resources are scarce, there will be no money for an army of data entry clerks and editors. The system must deal with this through good interfaces, smart business logic and community oversight.
3. Principle: The system and its community must attempt to create value propositions to entice other organisations and individuals to contribute what data they might already have. It's great that some groups may already have a database, but how useful is it if nobody but the 'insiders' can access it or cross reference to their own data? How much data on crop circles or big-foot sightings is simply ignored because those who might possess the data are precious about its release, or holding tight in order to profit? Where would Wikipedia be if nobody shared information? (They may be having funding issues right now but they are still an important cornerstone of the free internet) What good is a news article quoting an official 'spilling the beans' if it only exists as a rotting paper copy in somebody's basement? The system must encourage open involvement for the greater good.
4. Principle: the system must protect itself. In the off chance that certain conspiracy theories are true, there must be mechanisms to keep the data from being co-opted, destroyed or tainted. This implies leadership and precise control of sponsorship, if any. Understanding that the balance between leadership and community oversight has to be struck, who are the dedicated and serious people in the field who would lead such an effort? Are they interested enough in the advancement of the field(s) to get along with each other and set aside their desire to turn the thing into a book contract and lecture circuit?
5. Principle: the system must remain focused on analytics. What I mean here is that, while database design is fun and folks love to chime in about programming best practice, the focus must remain on outputs. All design should follow on from what the system is trying to achieve in terms of reporting and search-ability. A focus on analysis implies the system will have controls and smarts in place to identify duplication, provide the means to apply grading to data on-the-fly and offer whatever tools are available to identify correlations and patterns within the data.

How?
Well, that's not easy to answer. To start, discussion. Then to a prototype. Then to case building and promotion. These things tend to take on a life of their own once enough people are chipping in... thus the need for Principles and Goals. The initial technology is cheap or free and readily available. The talent is lying dormant. The time... well none of us have time anymore so maybe we have to be patient and grow it more slowly (but remember the difference between crystals grown at different rates!). Some of the better visualisation software will cost... maybe by then there's enough data and output to obtain some support from competitors of organisations whose names might pop out of the data?
I do know this: when all you can find is one ant carrying a crumb, you will remain living with ants. However, if you can track a hundred ants with crumbs as a line on a map, you can find the nest. And that kind of information has to be worth something to someone! (Maybe a bad analogy... I simple can't get rid the ants where I live.)
What do you think? Pipedream? Impossible? Faulty assumptions about human nature? Invitation to arrest and assassination under the 2012 NDAA? Let me have it. Here are the first round of Google hits I’ve spotted… there’s some nice work in here like the Model T was great work that led to the sports car. There are even some rudimentary attempts to create mapping. The proposition here is to take all this to the next level. I believe it's time and possible.
http://www.mufon.com/mufonreports.html
http://www.ufoconnect.com/
http://www.nuforc.org/
http://www.jtc-ufo.com/database/
http://www.larryhatch.net/
http://ufoaliendatabase.wikia.com/wiki/UFO-Alien_Database
http://www.ufoinfo.com/magonia/magonia.shtml
http://www.cufos.org/ufocat.html
http://www.greatdreams.com/ufos.htm
http://prufospolicedatabase.co.uk/1.html

 

[Gene Steinberg]

You've really put a lot of work into this concept. Thanks.

 

[LoremIpsum]

Thanks Gene. I am optimistic that at least a few enthusiasts, techies and/or seasoned field pros will be inclined to chime in.

 

[boomerang]

Lorum, looks like a very well thought out proposal. I'm guessing the main obstacle is familiar--lack of funding.

 

[LoremIpsum]

From one perspective, perhaps. But I might suggest, in keeping with certain capitalist mantras I have heard over the years, that 'there's always money out there'. Whilst this project may not pull the attention of venture capitalists like building a casino underwater, I'm confident the right pitch and demonstrable value will attract support.

I tend to feel the real issue is Time. Suitable software is free as are the skills to use the software or do the administrative and business development work. Hosting is dirt cheap, at least until you get the ball really rolling. What no one can spare much of these days is time. This puts any such endeavor into the position of either accepting very slow growth, begging, or waiting for luck. However, again I come back to some sage wisdom. I once asked a group of roughly 20 coworkers to contribute a mere 5 minutes a week to type up their knowledge of certain business processes into a wiki. Notwithstanding the need for some editing after two months or so we had produced more 'user manual' than a previous project had achieved for a five digit fee.

Naturally, the point I make here is that under the right circumstances even the issue of time can be overcome. Call it distributed computing, community consciousness or whatever. In the time it has taken me to write these posts I could have drafted a rough data model. If several dozen of us took 5 minutes per week, how far could we get? Far enough perhaps to entice others and further support?

Now I am not downplaying the role of funding. Somebody coughs up say $10k and we can certainly get something up quickly... And maybe that's important if we are competing for monetization of a market. But the open source approach tries to generate mutual value chains without this, and if successfully managed, can open new revenue models. Case in point: my workplace is about to pay a web agency good money for a website they will be building with an open source (free) content management system. And the only reason we aren't doing it internally is... You guessed it... Time.

So, until the concept is solid enough to justify writing a business case it should begin as a slow growing community project. What is most important right now is not money... Or even technology. Experience is what matters most as it is the detailed input of real users, the hardened paranormal field agents, that will identify what the system must achieve to be best than the others and useful.

So where are the researchers in this audience? What would you have such a database do for you?

 

[Voyager]

In terms of a database details from sightings need to be correlated. One problem with the UFO field is there is important information that we simply don't have. Fundamental questions that need to be answered before we can fully understand what it is we are dealing with. I can only relate to what I have observed firsthand here in Montana. People in other parts of the world have reported different types of craft that exhibit different behavior than what I have observed. What are the implications? One thing they all have in common, is they all use some form of field propulsion. A database would help sort out the common denominators and the differences.

 

[Blabyboy]

Dude! Where have you been for the last 8yrs of my life! I started another thread on this based of the first 10 mins of the Dec 18th show. Good to know that someone feels the same about this and suffice to say that I am on-board with this.

Cash is not and should not be the primary concern - it will come down to volunteers and data contributions and curation - you can have the best system in the world, but garbage in -> garbage out... assuming you get any garbage at all.

Whilst I agree that you need a strategy, I also think that you'll need to behave just like a start-up initially and JFDI to get something out the door for people to prod and poke at and get feedback to improve otherwise we're just stroking on a forum.

I would also suggest that to help protect itself, all datasets are pushed to a place such as github on a regular basis - this protects the data and allows anyone to get a copy and start to play with it, check it, interrogate it and provide new datasets. This also allows different developers in different languages to create derivative datasets or focus on a niche area.

I'm not sure if you're familiar with openstreetmap.org but I always see this a great model - starting from one person who mapped their local area and just put it up for people to use, and gradually more people wanted to improve it and did so. It goes from strength to strength in many of the areas you have listed. The model for distribution is interesting as well, but I can explain that later if interested, or you can look it up.

Infochimps have a 60K dataset file from NUFORC which could be a good starting point. (http://www.infochimps.com/datasets/60000-documented-ufo-sightings-with-text-descriptions-and-metada) and I'm currently scraping the uk pdf releases for UK information. That info chimps has some other interesting datasets for free e.g. Death Rates by Age, Sex and Race - 1940 to 2004 (http://www.infochimps.com/datasets/death-rates-by-age-sex-and-race1940-to-2004). Lay that over a map of sightings and see if you get any clusters at periodic intervals is one test that springs to mind.

In terms of closed data and value, one idea I thought of implementing was to allow investigators to input their data into the system and they could elect to make it private for a period of upto 6 months, after which it became public domain and therefore could be added by other investigators into their casefiles - the originator has the advantage to do something / generate publicity and it then is donated to the public good after that time.

The problem in the domain is that those sites referenced (and plenty of others) partially rely on eyeballs (and therefore ads) to generate revenue to keep them going, so yes, educating them to an OSS mindset might be a long game. One way to encourage would be to build a quick simple API that can be consumed into any website to at least raise awareness and link back to a central collecting hub. If the big players in the field donated some of their data then they could maybe take a greater steer in the strategy?

In some ways, what you're describing could just be seen as a web-ring with a focus on data.

As ever in this field, it won't be the technology or the data that might f**k it up, but the people and their egos. I just hope enough people 'get it'.

Happy to help.

Bb

 

[boomerang]

This has always been a great approach to the problem and it is better articulated here than most anywhere I have seen. From a layman's point of view, what about filtering data to be inputted? Some kind of credibility assessment algorithm would seem useful. Or perhaps that property would "parity check" itself as the overall scheme is constructed. For gosh sakes, don't ask me how. But there must be scads of talented people out there (and here) with good ideas.

At any rate, keep the momentum !

 

[Blabyboy]

From a layman's point of view, what about filtering data to be inputted? Some kind of credibility assessment algorithm would seem useful. Or perhaps that property would "parity check" itself as the overall scheme is constructed.

Any user supplied data would need to be filtered and depending upon how, and what data you collect, you could be quite specific in restricting 'bad' data into the system.

One way to provide a credibility assessment on the quality of the cases / observations would be to look at the amount of people that attached to it, and then check the quality of the people who have attached - similar to say Google in that they look at the sites linking to another site and it helps them to assess the quality of that site and whether it can be recommended. For example, someone with say a five star rating and attaching to a case, it would help to give a level of credence to that case that others might also rate. It's a trust based system which is what this whole area really needs and the wisdom of the crowd should, over time, pull out the better quality cases.

Bb

 

[LoremIpsum]

Great feedback! I am just dusting off after the holiday season and, sadly, a family loss right before. However, I will be back on this case shortly.

I am keeping local notes on ideas for layers based in geocoding, date and classification... Such as the recent Redfern show about bases/locations. I'm also intrigued by Linda Moulton Howe's comments on data she holds related to her areas of interest. Time lapse mapping of hotspots across seemingly unrelated genres may be pushing the envelope of what a web app can do, but I'm sure I'm not the only one it excites.

More...

 

[LoremIpsum]

Hello All,

Just a quick update. I am still shaking off the loss of my Mother pre-xmas but hope to be back on the case over the next few weeks. The idea of getting a small project website together for this has emerged in order to better manage features and issues etc. I'll look into this shortly. I've also found another reference site UFOdB.com - UFO/Alien Visual Evidences Database | Picture | Video | News. I am keen to hear ideas regarding what outputs could be developed first/quickly in order to interest the pros out there that I suspect are already aware of many sophisticated database or classification efforts.

Ta

 

[LoremIpsum]

So let's move on...

There has been some extremely detailed and valuable feedback thus far... The engine appears to be warming up. As I write from the bus ride to work today a few things occur to me.

First, if I were approaching this challenge in a corporate environment I would be capturing the scope and requirements, as well as aspects of the business case, in a formal document. The forum is a great place for conversation but I feel it critical to extract from here into a framework. I am also receiving a number of direct messages from other members that contain great questions and suggestions which need to be captured in a centralised manner and made available to everyone. At the moment I am playing with a Wordpress site to see if it can take on the feel of a reference and handle issue lists.

Second, despite knowing better at this early stage I am wondering about the format of the eventual product. I have always contemplated an online system supporting features for data capture, complex search, bulk export and visualisation. However after looking a bit further at online technologies for mapping I am finding that what exists falls short of what I want achieved. It's not enough to plot points on a map... What we desire is more akin to what we can perform in tools like arcgis... Area calculations, polygons, massive layering, time scale analysis etc etc. it may be that such advanced visualisation may have to remain a value add off the core system as it does not seem possible to offer that over a web page to everyone.

Third, taxonomy, taxonomy, taxonomy. Entity model. Process mapping. Typing for persons, events, craft and creatures. Meta groupings of attributes... Temporal, geolocational, meteorological, key proximities, behavioural, vectorial, physical trace, links to reference media, measurements and readings, (breath), rating mechanisms, versioning, key contacts, copyright and privacy, contextual aspects, officiated explanations, linguistic variations and cultural annotations...

Mercy.

 

[stonehart]

So let's move on...

There has been some extremely detailed and valuable feedback thus far... The engine appears to be warming up. As I write from the bus ride to work today a few things occur to me.

First, if I were approaching this challenge in a corporate environment I would be capturing the scope and requirements, as well as aspects of the business case, in a formal document. The forum is a great place for conversation but I feel it critical to extract from here into a framework. I am also receiving a number of direct messages from other members that contain great questions and suggestions which need to be captured in a centralised manner and made available to everyone. At the moment I am playing with a Wordpress site to see if it can take on the feel of a reference and handle issue lists.

Second, despite knowing better at this early stage I am wondering about the format of the eventual product. I have always contemplated an online system supporting features for data capture, complex search, bulk export and visualisation. However after looking a bit further at online technologies for mapping I am finding that what exists falls short of what I want achieved. It's not enough to plot points on a map... What we desire is more akin to what we can perform in tools like arcgis... Area calculations, polygons, massive layering, time scale analysis etc etc. it may be that such advanced visualisation may have to remain a value add off the core system as it does not seem possible to offer that over a web page to everyone.

Third, taxonomy, taxonomy, taxonomy. Entity model. Process mapping. Typing for persons, events, craft and creatures. Meta groupings of attributes... Temporal, geolocational, meteorological, key proximities, behavioural, vectorial, physical trace, links to reference media, measurements and readings, (breath), rating mechanisms, versioning, key contacts, copyright and privacy, contextual aspects, officiated explanations, linguistic variations and cultural annotations...

Mercy.

Holy Sh*t
This could get very interesting. I am not an IT tech just and Audio Engineer but I do appreciate your enthusiasm.

 

[Han]

I think you have a very good concept here some ideas or suggestions that occur to me.

I think that Hoaxes, misidentifications and solved cases should be included in you "Database" as they can be extremely usefull for comparison.

I also think establishing a "vocabulary" or "language" would be advantageous. (for example people often use the term U.F.O. when talking about a flying saucer or disk.)*




*please add words to this list ((Parawiki Idea (help pls) | The Paracast Community Forums))

 

[bbridges]

I think you have a very good concept here some ideas or suggestions that occur to me.

I think that Hoaxes, misidentifications and solved cases should be included in you "Database" as they can be extremely usefull for comparison.

I also think establishing a "vocabulary" or "language" would be advantageous. (for example people often use the term U.F.O. when talking about a flying saucer or disk.)*

Has anyone suggested contacting Jacques Vallee about this project? His experience in developing models and categories, as well as his computer skills could be helpful.


*please add words to this list ((Parawiki Idea (help pls) | The Paracast Community Forums))

 

[Han]

"Has anyone suggested contacting Jacques Vallee about this project? His experience in developing models and categories, as well as his computer skills could be helpful."*

*said by "bbridges" see previous message.

 

[bbridges]

"Has anyone suggested contacting Jacques Vallee about this project? His experience in developing models and categories, as well as his computer skills could be helpful."*

*said by "bbridges" see previous message.

Hell; why don't we just ask the NSA if we could use their version?

 

[LoremIpsum]

I decided you were right:

Hello,

I'm not sure if I am directing this to the correct department however I have a somewhat unusual enquiry and I hope I might be able to obtain your assistance or at least a referral to the correct area.

Outside of my career work I contribute my technical skills as a hobby to areas of personal interest... in this case to the field of unexplained aerial phenomena (yes, that does include 'UFOs' though is not limited to them). My interest in this area does not extend to research or participation in any socio-political discussion, but rather is focused on the development of information cataloguing systems that aide information collection and analysis.

I am specifically seeking to leverage any existing classification systems or taxonomies that may be in use within professional, academic or government institutions for the purpose of collecting data related to aerial phenomena.

Is it possible to find out if your agency utilises such a thing and whether the taxonomy used might be available for public consumption?

Many thanks.

 

[Goggs Mackay]

Amen.

Absolutely. If I had any worthwhile help, I'd offer it freely...

The one thing that is hard to attack is raw data, in fact it is the very lack of raw data that often hampers any efforts in the field of the 'not too normal'. Now, some debunker types will not care even about vast amounts of data but they are a lost cause anyway. Open minded skeptics and generally open minded people in addition to those who are already interested would be interested in your idea. I most certainly would be.
Great post Loremlpsum.

 

[LoremIpsum]

Actually, can I ask all who are interested to email that message to organisations they feel might have something and post the replies here? I've just sent it to several contact points at NSA.