Kudos to the Organizers of the First International Virtual Conference on Ukraine

Today we mark Ukraine's Independence Day (33 years of independence) and the past two days I had the distinct pleasure of taking part in the first International Virtual Conference on Ukraine. It was excellent and I wanted to express my thanks to Professors Almas Heshmati, Lars Hartvigson, and Olena Nizalova for putting together such a timely and interesting program. It was laudable how the scheduling was done with speakers and participants from Hawaii, California, Texas, Massachusetts, multiple European countries, including Ukraine, as well as Vietnam, Australia, and Morroco!

Jonkoping International Business School (JIBS) in Sweden was the host of this virtual conference, in collaboration with its partners.

I took a snapshot of the closing session, which is posted below.

Each day, the conference began at 3AM, my time; but, despite this, I thoroughly enjoyed the sessions that I managed to join and, of course, it was an honor to present our latest research with my PhD student Dana Hassani and Kyiv School of Economics colleagues Oleg Nivievski and Pavlo Martyshev.

The presentation was based on our paper, noted above, which will be published any day now in the INFORMS journal Transportation Science. The slide deck of our presentation can be downloaded here. I was delighted that all the co-authors of our paper were present! Plus, I enjoyed "seeing" colleagues in Ukraine, with whom I have also published: Elena Besedina of the Kyiv School of Economics and Myroslava Kushnir of the Ukrainian Catholic University in Lviv.

I also thoroughly enjoyed being on a panel with: 

Pham Khanh NAM, UEH University Ho Chi Minh City, Vietnam

Hans LÖÖF, Royal Institute of Technology (KTH), Sweden

Vesa KANNIAINEN, University of Helsinki, Finland

Mykola TKACHENKO, CEO of SOE Prozorro (Transparency), Ukraine.

Yuriy BILAN, regrettably, could not make our panel, as scheduled.

A remarkable amount of information was exchanged and the discussions were terrific.

The keynote talks by Roman Sheremeta and Yuriy Gorodnichenko were excellent! I wish that all presentations could have been recorded and posted since with the different time zones it was challenging to hear all in real time.

Themes that resonated throughout included: the resilience of Ukrainians; the fact that Russia's war on Ukraine is a massive threat to global security; Ukraine needs immense support from its allies; economic & military aid to Ukraine is the best investment; agriculture and IT continue to be shining lights in Ukraine whereas steel production has essentially collapsed; investing in transportation routes is critical and keeping maritime routes safe and efficient; human capital must be brought back and nurtured; security is of the utmost importance; damages are immense to educational institutions, healthcare, critical infrastructure (energy, transportation, logistics, supply chains), and the environment, and so much more!

I was impressed how much researchers and practitioners and even PhD students who took part want to help. There was emphasis on the need to get research results in front of policy and decision-makers more quickly.

Information on this conference can be found here. 

The full program has been posted here.

Some Great Books I am Reading this Summer

This has been quite the eventful (and wonderful) summer. After a glorious month at the University of Gothenburg in Sweden we returned to Amherst and, in just the past few days, we took part in the Isenberg Business Leadership Awards gala in Boston and also celebrated my doctoral student's Dong "Michelle" Li's completion of her PhD and acceptance of a tenure-track Assistant Professorship job offer. Michelle had 3 such job offers and made her decision after careful deliberations.

I am so proud of Michelle, who is my 18th PhD student, whose dissertation committee I have chaired. Below we are together after our dinner celebration at Judie's this past Monday evening.

This summer, in addition to spending a month in Sweden, doing research, co-organizing a conference in Greece, which takes place next week, and making preparations for the EURO conference, plus the fact that I am also writing a book, I have had the pleasure of reading some great books (all nonfiction and appearing in the photo below).

And to the authors and editors of these books there is a neat "connection."
 

All the volumes in the photo above I received from the author or editor, except for the first one, The Handbook of Global Logistics, edited by James H. Bookbinder, which I got from the publisher, Springer, since I had the pleasure of reviewing this 21 chapter handbook for the European Journal of Operational Research, and the review is now in press. Bookbinder's inspiration for the handbook came out of a special issue of  Transportation Research E, volume 41(6), 2005, on Global Logistics, that he guest edited. I admit I had a  paper in that special issue: Global Supply Chain Network Dynamics with Multicriteria Decision-Making Under Risk and Uncertainty,, Anna Nagurney and Dmytro Matsypura, Transportation Research E 41: (2005) pp 585-612. I did not, however, have an invited contribution in the handbook that I reviewed.

I read Bookbinder's magnum opus cover to cover and prepared the review of it while in Sweden (working many hours on it). I enjoyed the handbook a lot since the theme is of great interest to me and I also teach a course on Transportation and Logistics. The chapters are by different authors, some of whom I know very well.

The book, Forecasting Urban Travel, by Davie E. Bouce and Huw Williams, I am now reading and I have read several chapters. It is over 600 pages and is a historical rendition of transportation forecasting over a sixty year period from US and UK perspectives. It is a massive, extensive piece of scholarship. The book is my nightly reading now as a reward for a good day. I am eagerly looking forward to Chapter 7 on Transportation Network Equilibrium. Transportation Network Equilibrium is a topic that was my first research passion (and that my dissertation advisor, Stella Dafermos at Brown University, contributed so much to). It was  one of the themes of my dissertation at Brown University. I did get a preview of Chapter 7, since David Boyce, who has been a mentor and friend for many years, wanted my comments and feedback on it. This chapter, alone, I believe, is worth buying the book for since so many operations researchers and transportation scientists are noted there and their contributions. I am so honored to be cited in this book!

The book, Telecommunications Network Economics, I am taking with me on my upcoming European 3 week sojourn. Patrick Maille, one of the authors (with Bruno Tuffin), sent me a complimentary copy. Given that my great collaborator, Tilman Wolf, and our Co-PIs on a big NSF grant, Jim Griffioen, Ken Calvert, Rudra Dutta, and George Rouskas, with our students, have been working on the Future Internet Architecture project known as ChoiceNet, for 4 years, now, this book should be a great read. And, yes, I peeked -- I am cited - specifically, my Supply Chain Network Economics book is, which I wrote while at Harvard.  Patrick Maille  is an Associate Professor at Telecom Bretagne (part of Institut Mines-Telecom) in France. This year he is visiting Jean Walrand’s team at UC Berkeley (I am a huge fan of Walrand's work). Maille has asked me to serve on his habilitation (Habilitation à Diriger les Recherches, in French)  committee, that he will defending  in Rennes, France in October. I have agreed.

A habilitation, according to Wikipedia, is the highest academic qualification a scholar can achieve by his or her own pursuit in many countries in Europe, Central Asia, and Egypt^... Earned usually after obtaining a research doctorate, such as a PhD, the habilitation requires that the candidate write a professorial thesis (or habilitation thesis) based on independent scholarship.  I will also be reading his habilitation thesis this summer!

The book, Innovation Spaces in Asia, edited by Maurren McKelvey and Sharmistha Bagchi-Sen, I received as a gift from Maureen McKelvey at a lunch in Sjobaren in Gothenburg, Sweden 2 weeks ago. Maureen is a great friend and an exceptional scholar and served, for several years,  as the Dean of the Graduate School at the School of Business, Economics and Law at the University of Gothenburg. Her fellow co-editor, Sharmistha, was also a Visiting Professor at the school with me. Given the excitement surrounding innovation and entrepreneurship (and the Isenberg School received a gift of $10 million to fund the Berthiaume Center for Entrepreneurship, for example), this book I will be reading after returning from Europe.

You may have noticed that the bottom set of three books featured above in the photo has a Swedish connection. I already mentioned the book co-edited by McKelvey, which I carried back from Gothenburg less than 2 weeks ago. I also carried the volume edited by Jonas Floden, Sustainable Intermodal Biofuel Transport, and the dissertation, Essays on Performance Management Systems, Regulation and Change in Swedish Banks, by Viktor Elliott, who successfully defended his dissertation while I was in Gothenburg and I went to the party. The defense itself was in Swedish so I waited in my office for it to be over with and then we celebrated with champagne (see photo below)!

I have read parts of both of these books and have enjoyed them a lot. Jonas and I are obsessed with sustainability and supply chains and we have co-authored several papers together. At the EURO Conference in Glasgow, Scotland, in July I will be presenting our paper, Fashion Supply Chain Network Competition with Ecolabelling, Anna Nagurney, Min Yu, and Jonas Floden, Sustainable Fashion Supply Chain Management: From Sourcing to Retailing, T.-M. Choi and T.C.E. Cheng, Editors, Springer (2015) pp 61-84. Viktor's friendship I will always value. He was always ready and willing to give me lots of advice during my 4 year tenure as a Visiting Professor at the University of Gothenburg (including great restaurant recommendations) and for that I thank him!

Congratulations to all the authors and editors of the above books as well as to the contributors of the edited volumes! There is nothing like a great book!

Happy Summer Reading!

And to those who are writing book, Happy Writing!

Entrepreneurial Opportunities and Innovation Thanks to the National Science Foundation (NSF)

Robert Shiller, the Sterling Professor of Economics at Yale University, wrote a wonderful OpEd in last Sunday's New York Times: "Why Innovation is Still Capitalism's Star," in which he spoke about the company he founded with Karl Case of Wellesley College (I used to attend seminars at Harvard when I was a Visiting Scholar and Associate Professor at the Sloan School and School of Engineering, respectively, at MIT, at which Karl would be present). The company was sold but its "products," now known as the S&P/Case-Shiller Home Price Indices, are very often used and referred to.

In the OpEd, Shiller writes that his business made its mark without any help from the government but then writes the following:

Long before I started any commercial ventures of my own, I received some federal government support — in the form of National Science Foundation research grants, awarded to me decades ago as a young professor. They allowed me to do research, and though it was not directly related to my later business endeavors, the process developed my expertise and reinforced a sense of entrepreneurial opportunity.

These grants were awarded competitively, based on the quality of the proposals, and gave me experience with a system focused on creating opportunities for those who try hard. Later, from 1983 to 1985, I evaluated others’ proposals when I served on the foundation’s panel for economics. Observing the process from the government side convinced me that the foundation really works. Maybe it’s because the panelists are chosen from successful scientists, who serve anonymously out of public spirit.


The first year that I was at MIT, 1989-1990, I was funded by the National Science Foundation's Visiting Professorship for Women, and was based at the Center for Transportation in the Department of Civil Engineering (Building 1 for those of you familiar with MIT).  Besides teaching a course there and doing a lot of research, I also organized a Women in Operations Research (OR) Speaker Series, which took place at MIT's Operations Research Center, located on Amherst Street (which made me feel at home). I was following in the footsteps of my dissertation advisor at Brown University, Professor Stella Dafermos, who had also had an NSF Visiting Professorship at MIT,  a few years prior to me. My readers know that she was the second female in the world to received a PhD in OR!


The receipt of  that NSF grant, plus 3 more, early in the new millennium, including a large one with two females: professors,  June Dong and Patricia Mokhtarian, gave me the confidence needed to take risks with my research, and engendered an entrepreneurial spirit.

That large NSF grant, Decentralized Decision-Making in Complex Network Systems, along with two AT&T Industrial Ecology Fellowships, enabled me to establish the Virtual Center for Supernetworks at the Isenberg School of Management in 2001.

An additional stimulus behind the Supernetworks Center was the publication of  my book, with June Dong, Supernetworks: Decision-Making for the Information Age.

In 2003, under the leadership of our great Dean at that time, Dr. Tom O'Brien (with whom I had such a nice conversation the other day), I was given lovely space to create the Supernetworks Laboratory for Computation and Visualization. Our new Dean, Dr. Mark Fuller, now holds a Chaired Professorship in Tom's name.


And, just this past week, I received the letter from our Provost, James Staros, and our Vice Chancellor for research and Engagement, Mike Malone, that the UMass Amherst Evaluation Committee for Centers & Institutes has recommended the continuation of the Virtual Center for Supernetworks for another 5 years. The evaluation cited many of the activities of the Center and its Associates, which have even included undergraduates conducting research under NSF's Research Experiences for Undergraduates (REU) program.


As I have written before, in the context of Steve Jobs, risk-taking is essential to innovation, and support, such as that provided by NSF, gives females concrete evidence and financing that they can lead and even establish a center. And, now, we are involved in another truly thrilling NSF research project, Network Innovation Through Choice.

Thank you, NSF, for supporting scientific research without boundaries and for recognizing also the contributions of females to scientific innovation and discoveries!

Business Professors as the Ultimate Entrepreneurs

From investors to educators and the media there is a storm of attention being paid to entrepreneurship  with initiatives ranging from the much-publicized Venture for America, founded by a fellow Brown University alum, Andrew Yang,  to newly formed centers, even here at the Isenberg School of Management, which is headed by my colleague, Professor James Theroux (he is the brother of the award-winning travel writer, Paul Theroux). There are also well-established entrepreneurship programs, typically, at business schools.

The Center for Entrepreneurship is only of several centers at the Isenberg School. 

For example, back in 2001, I founded and continue to direct  the Virtual Center for Supernetworks.
 
So what is entrepreneurship?

Consider, as pointed out by Brett Nelson in Forbes, the following two definitions of “entrepreneur”: the first one from in Merriam-Webster’s dictionary, and the other from Dictionary.com:

• Merriam-Webster: “one who organizes, manages and assumes the risks of a business or enterprise.”

• Dictionary.com: “a person who organizes and manages any enterprise, especially a business, usually with considerable initiative and risk.”

We both agree that the latter definition, which considers any enterprise ... as being the relevant one today.

To survive and thrive in academia, including business schools, one must be entrepreneurial. One can't just sit back and leave the status quo, since the environment in both business and academia is much too dynamic, evolving, and changing. Entrepreneurship goes hand in hand with innovation.

Without entrepreneurship and innovation on the part of faculty:

• No new courses will be offered.
• No new programs and majors started.
• No new partnerships, including global ones, initiated.
• No frontier research conducted.
• No centers established nor grants obtained.
• An academic institution will fail to evolve and thrive.

Business professors, in particular, are great at seeking out new opportunities -- it is Darwinian -- if you don't take risks and initiate, you rot -- and the students and institutions suffer.

We are also quite good (thanks to the professors that we have had and, for some of us, actual industrial experience) in allocating our scarce resources in as optimal of a fashion as possible in order to get the highest return with some risk involved, of course. I think that this is part of our DNA.

If you don't have courage and take risks, nothing valuable or interesting happens -- Nothing ventured -- nothing gained.

Of course, companies have also been formed by business school profs and we in operations research and the manageement sciences can name a few easily off the tops of our heads.

As importantly, the students that we have educated and inspired go on to organize and manage their enterprises through their initiatives!  What is more entrepreneurial than an excellent doctoral dissertation?!

Preparing a Graduate Commencement Speech -- From the Isenberg School of Management to the School of Business, Economics and Law in Sweden

Now is the time for college graduations and, in Massachusetts, over the past several days, we have been the scene for numerous ones -- from the UMass Amherst graduations to graduations in Boston and to even the Smith College graduation last weekend at which Arianna Huffington gave the commencement address. And Oprah Winfrey will be speaking at Harvard University's commencement on May 30.

Last, year, I was honored to give the commencement address at the Master's degree graduate commencement ceremony at the University of Gothenburg's School of Business, Economics and Law on June 14, 2013. My speech was on Life as a Network and it can be accessed here.

A writeup on the speech appeared in the UMass Amherst's In the Loop.

When I received an invitation a few months ago from Dean Maureen McKelvey of the University of Gothenburg to give this year's commencement speech, I immediately agreed.

Giving a commencement speech in the US is one thing -- giving a commencement speech in another country is a truly special experience.

I had the great experience of giving the commencement speech at the Isenberg School's undergraduate commencement in May 2003, which took place at the Mullins Center, so I can say that I have given such speeches on 2 continents.


This will be my fifth visit/stay in Gothenburg, since 2012, and,  as a Visiting Professor of Operations Management there, through its unique Visiting Professorship Programme, Gothenburg, Sweden has become my second home.

I will be delivering my commencement speech on June 12, 2013 in Gothenburg and my theme this year will be Leadership.

I will speak on what makes a great leader -- one of many themes that I have been writing about on this blog and will also bring personal experiences into the text.

I plan on emphasizing resiliency, creativity, communications, diversity, and integrity, and, as I did last year, I will share with you photos from the ceremony.

Risk-Taking, Innovation, and Steve Jobs

The sun has not yet risen, the power has not failed, and we are awaiting Hurricane Irene, which is to hit landfall in our part of the Northeast sometime today. Hurricane Irene and emergency preparedness have been the center of attention with mass transit shutdowns in Philadelphia, NYC, New Jersey, and Boston, and thousands of flights cancelled and even airports closed as the hurricane barrels up the northeast corridor. Manhattan was eerily quiet yesterday as my uncle, who is in his 90s, related to us, after his walk to Lincoln Center and back to his apartment.

During this period of preparing for Hurricane Irene, and, unlike when the tornadoes hit Massachusetts on Jun 1, 2011, there was sufficient warning, the announcement was made, in case you missed it, that Steve Jobs, the CEO of Apple Inc. had stepped down, but had asked to remain on the Apple board as its chairman. He had been fighting pancreatic cancer for two years and had had a liver transplant and the best wishes and accolades streamed from around the globe from leaders, executives, and consumers of Apple's products from the iPod to the iPhone and iPad. Jobs' attention to detail and sense of aesthetics are legendary. He is also the holder of 313 patents according to The New York Times.

Jobs is the consummate innovator and innovation is an elusive talent and quality that has generated much interest in both industry and academia and is essential for economic growth and prosperity. In fact, presently, we at the Isenberg School, are looking to fill a chaired faculty position as the Isenberg Professor, who is to focus on innovation. Innovation has even attracted the attention of nations and, according to John Kao, an innovation expert, and as reported in The Times, the raw materials for innovation, which other nations may be surpassing the US in, include:

• government financing for scientific research,
• national policies to support emerging industries,
• educational achievement, engineers and scientists graduated, and
• even the speeds of Internet broadband service.

What the US does have, however, and what other countries may lack, and which is essential for a climate for innovation, Mr. Kao notes, is a social environment that encourages diversity, experimentation, risk-taking, and combining skills from many fields into products that he calls “recombinant mash-ups,” like the iPhone, which redefined the smartphone category.

To learn how to become a great innovator, we need look no further than Steve Jobs and, as academics have pointed out: the five traits that are common to the disruptive innovators or what makes up The Innovator's DNA: questioning, experimenting, observing, associating, and networking to search for new ideas. Ceaseless curiosity and willingness to take risks make up their genetic code.

Thank you, Steve Jobs, for showing us how it should be done -- even after your firing from Apple, you rose to lead the company to new heights.

A New Business Era for Analytics and Why Math Matters

According to The New York Times, there is a shortage of graduates with math skills, who can analyze the massive amounts of data that are now available for businesses (and other organizations).

Economists are saying that we are entering a new era, similar to the Industrial Revolution, in which the access to huge volumes of data can spur innovation, new products, and new business models.

McKinsey & Company issued a new report, Big Data: The Next Frontier for Innovation, Competition, and Productivity, which bodes well for those of us who, as business professors, teach the quantitative subjects, such as operations research / management science, as well as for our students.

Clearly statistics and experimental design and data mining are essential for filtering out the data streams but, as important, is knowing what to do with the data and how to optimize our business (and other) processes accordingly. We can't just sit and analyze the past but need to work on managing the world's resources for a safer, more secure, and sustainable future. Math matters and, besides, not only is it so much fun but doing and applying math is also rewarding and useful!

Perhaps the US will finally wake up and realize how essential math education is to the present and the future of our country and its very competitiveness.

Diversity for Scientific Innovation

Alan L. Leshner, the Chief Executive Officer of the American Association for the Advancement of Science (AAAS), wrote an excellent commentary in the latest issue of The Chronicle of Higher Education, entitled: "We Need to Reward Those Who Nurture a Diversity of Ideas in Science."

In his commentary, he argues that, by increasing the diversity of the scientific human-resource pool, we will inevitably enhance the diversity of scientific ideas, since, by definition, innovation requires the ability to think in new and transformative ways. Moreover, many of the best new ideas come from new participants in science and engineering enterprises, from those who have been less influenced by traditional scientific paradigms, thinking, and theories than those who have always been a part of the established community.

I completely agree with Mr. Leshner that we need to enhance diversity in terms of the inclusion of women and minorities, and reading his thoughtful piece truly brightened my day since it shows that others recognize the need. I would, however, add that we also need to break up disciplinary silos and to work on creating diversity across disciplines, and only then, will we be able to, as President Obama stated in his State of the Union address (coincidentally, I was in DC then for the Transportation Research Board (TRB) meeting), "out-innovate, out-educate, and outbuild the rest of the world."

Indeed, creativity is found everywhere and needs to be nurtured.

Finally, as Leshner so well elucidates, universities need to recognize faculty that work on increasing participation in science and engineering and even notes that universities should also pay tribute to such faculty through public acknowledgment of such successes through press releases, articles in university publications, etc., to send a clear message of support. Personally, working with a diverse body of students is the greatest reward since we all learn so much from one another -- how boring the world would be if there was perfect homogeneity!

Isenberg Professorship for Innovation and Entrepreneurship

The Isenberg School of Management at UMass Amherst has received approval for a search for the Eugene M. Isenberg Professorship for Innovation and Entrepreneurship. The closing date for this chaired professorship is April 1, 2011 or until the position is filled. The Search Committee consists of Faculty from the Operations Management, Management, and Marketing disciplines since it is expected that the holder of this professorship will be in one of these areas, and the VCRE at UMass Amherst, Professor Michael Malone.

A link to the position description on the Isenberg School site is here. Ads for this position are expected to be placed in The Chronicle of Higher Education, ORMS Today (online), ASEE Prism, among other venues.

The previous holder of this chaired professorship was Dr. Soren Bisgaard, who passed away on December 14, 2009. Other holders of Isenberg chaired professorships are Dr. Michael "Mike" Malone, who is the Isenberg Distinguished Professor of Engineering, in addition to serving as the Vice Chancellor for Research and Engagement, and Dr. Lawrence M. Schwartz, who is the Isenberg Professor in Integrative Science, and who was named this past year. These professorships are endowed by Ronnie and Eugene M. Isenberg, after whom the Isenberg School of Management is named.

Forensic Engineering and Learning from Disasters

I have just about finished reading Henry Petroski's book, "The Essential Engineer," which was published in 2010. It is filled with excellent ideas about why engineering is different from science and how science and engineering and their practitioners and innovators must work together to address the grand challenges today from renewable energy to reducing vulnerability and even to securing cyberspace.

Henry Petroski is a professor of civil engineering and history at Duke University.

Coincidentally, in today's New York Times, there is an article by William Broad, "Taking Lessons from What Went Wrong," which begins with the eye-catching sentence: Disasters teach more than successes, with the overall thesis that disasters can spur innovation. The article includes an interview with Petroski and a graphic photo of the BP oil rig disaster.

Technological feats that define the modern world are sometimes the result of events that some might wish to forget, from the collapse of the Tacoma-Narrows bridge in 1940 due to winds (with no lives lost), to the collapse of the Minneapolis bridge in 2007 (with 13 lives lost), to the sinking of the Titanic on its maiden voyage (with over 1,500 deaths, some due to hypothermia), and even the World Trade Center disaster (with approximately 3,000 deaths). Now we are all reeling from the BP oil rig disaster with ups and downs on almost a daily basis as to progress or lack thereof regarding the spill containment and the propagation of the massive effects on the environment and affected economic sectors and regions.

According to Petroski, disasters are “ a great source of knowledge — and humbling, too — sometimes that’s necessary.” He is also the author of “Success Through Failure,” a 2006 book. “Nobody wants failures. But you also don’t want to let a good crisis go to waste.”

I had written earlier on this blog about forensic accounting and we had even hosted Dr. Brian Levine who spoke on his research on the forensic investigation of the Internet and mobile devices.

Our modern era demands a new area of expertise -- that of forensic engineering, which should clearly have risk management and policy analysis as essential constructs to assist in lessons learned (so mistakes do not get repeated in the future).

Interestingly, Petroski uses as vivid examples in both of his two books noted above the challenges of engineering design in the context of bridge design. He considers bridge designers as very creative individuals who develop mental constructs of a bridge, combined with aesthetics, and then design mathematically the functional structure, which, I might add, should last for many years and support the weight of numerous vehicles.

My uncle, Stanley Jarosz, is an award-winning bridge designer, who, although he is almost 92 years old, still works several days a week at an engineering firm. He is one of my greatest inspirations and an exceptional role model and gentleman (who, I might add, is also a big opera aficionado).

I had the pleasure recently of seeing my uncle and my terrific cousin, Andrew (who, I might add, is a fellow Brown University grad), in NYC. I discussed Petroski's "The Essential Engineer" with my uncle and noted Petroski's almost mystic adulation of bridge designers.

Solving the grand challenges faced by our civilization will require the cooperation and the working together of our best, creative minds, as well as capturing, in a quantifiable and rigorous manner, the risk associated with the resulting innovations.