The Defense Advanced Research Projects Agency (DARPA), the defense department group that brought us the ARPAnet, the nascent beginnings of the Internet, in 1969, has moved into the territory of Google, Bing, and Yahoo, and is creating a search tool called Memex. The problem with the commercial search engines is that they do not catalog the Internet underground known as the Dark Web or Deep Web. This is primarily because they are commercial services, and due more to a lack of interest than any lack ability.
DARPA is initially looking to focus the resources of Memex on the problem of human trafficking as it is supported on the web. On their website they give the reasons for this new search technology as this:
“Today’s web searches use a centralized, one-size-fits-all approach that searches the Internet with the same set of tools for all queries. While that model has been wildly successful commercially, it does not work well for many government use cases. For example, it still remains a largely manual process that does not save sessions, requires nearly exact input with one-at-a-time entry, and doesn’t organize or aggregate results beyond a list of links. Moreover, common search practices miss information in the deep web—the parts of the web not indexed by standard commercial search engines—and ignore shared content across pages.
To help overcome these challenges, DARPA has launched the Memex program. Memex seeks to develop the next generation of search technologies and revolutionize the discovery, organization and presentation of search results. The goal is for users to be able to extend the reach of current search capabilities and quickly and thoroughly organize subsets of information based on individual interests. Memex also aims to produce search results that are more immediately useful to specific domains and tasks, and to improve the ability of military, government and commercial enterprises to find and organize mission-critical publically available information on the Internet.”
This is an open source project, and tools are being released as they become available. Certainly this search capability is going to be of interest to police and government agencies, but ultimately it could fine its way into the search toolbox of business and other non-governmental organizations. If you are interested in looking at what has been released, go to the DARPA Open Catalog.
More info:
Share
MAY
About the Author:
I am a cybersecurity and IT instructor, cybersecurity analyst, pen-tester, trainer, and speaker. I am an owner of the WyzCo Group Inc. In addition to consulting on security products and services, I also conduct security audits, compliance audits, vulnerability assessments and penetration tests. I also teach Cybersecurity Awareness Training classes. I work as an information technology and cybersecurity instructor for several training and certification organizations. I have worked in corporate, military, government, and workforce development training environments I am a frequent speaker at professional conferences such as the Minnesota Bloggers Conference, Secure360 Security Conference in 2016, 2017, 2018, 2019, the (ISC)2 World Congress 2016, and the ISSA International Conference 2017, and many local community organizations, including Chambers of Commerce, SCORE, and several school districts. I have been blogging on cybersecurity since 2006 at http://wyzguyscybersecurity.com