There is an abundance of information security conferences out there. While the industry is drowning in these conferences, a lot of them are producing more noise than value. Increasingly, people are realizing that even a moderate security conference is a profit center. We need fewer conferences that are more topical and offer more value, whatever the price. In addition to the frequency of conferences, most of them are doing the same exact thing. There is a serious lack of creativity and forward-thinking. It was only the last few years that saw a couple conferences dedicate entire tracks to defensive security.
I have been attending security conferences for almost 20 years now. Based on my experience, as well as being on several CFP review teams, there are many aspects I want to see in the future.
- More talks or entire tracks dedicated to sociology and human sciences, as relates to the security world. We see this from time to time, usually in passing regarding security awareness or phishing. Attacker profiling is a stronger use, but most talks are over-simplified and don’t cover new ground.
- Talks on law and policy are more frequent lately, but they don’t seem to do much good. In the recent DEF CON 21 CFP review, we received many talks that focused on law and/or policy. There was one trend that emerged between all of them; no practical information on how the average person can truly make a difference. Sure, write your congress critters, stay informed, and all the usual advice. That hasn’t worked in the past. What else do you have?
- Heckling should be encouraged. Several years ago, DEF CON changed to where questions or comments were not allowed during talks. The years prior, if a speaker said something that was not factual, you could quickly call them on it. It gave the audience a chance to see the error with minimal interruption. Now, questions are done after the talk, in a separate room, away from the audience. If a speaker says something inaccurate, the audience leaves thinking it was factual. This is a disservice to the attendees. Speakers must be kept honest.
- Continuing that theme, all talks should have a mandatory 5 minute Q&A session at the least. It is rare that a speaker is so decisive and thorough as to leave no questions. If an audience member wants to debate a point or call them on bullshit, they get an opportunity to do just that.
- More lightning talks, with a twist! Having 3 presentations in an hour gives more researchers a chance to share their progress and ideas. It gives a brief platform for them to find others that may want to help, or get ideas for moving forward. The twist? A gong. If a talk is bad or going nowhere, don’t even give them their 15 or 20 minutes. Gong them off the stage and let the next lightning talk start.
- Most conferences solicit talks (the CFP), have a review team decide which are worthwhile, and create a schedule. It would be nice to see conferences follow this process to weed out the crap, but then put all good talks up for community vote. Based on the feedback, use it to determine what the masses want to see and then build a schedule off the higher voted talks.
- Speakers should not only explain why they are presenting, they should justify why they are the ones giving the talk. Not a general resume with 20 years of security experience either. What specifically have they done that warrants them giving this talk. Pen-testers with a few years of experience should rarely give a talk on pen-testing or social engineering, unless they truly have groundbreaking material. They should be required to make their slides available shortly after the convention. The slides should properly reference and footnote prior work, source images, and give credit to what influenced them.
- Conferences should solicit feedback from the audience, and give it to the speakers so that they may improve their talks in the future.
These are but a few ideas for improving conferences. Have your own ideas? Leave a comment!
10 responses to “Building a better InfoSec conference…”
There has been a trend to get new people on the stage at some cons. Even creating entire newbie tracks for them. The reasons are that people are tired of seeing the same old people on stage, opportunity to inject ‘new blood’ and to give people experience with speaking. The problem I have with this is that the quality of the talks are often lacking, both in content and presentation skill. If the content is new/relevant/exciting then I can forgive the presentation quality but that is seldom the case. Now instead of having a familiar but engaging speaker with a relevant topic you end up with a introductory level talk given by an ineffectual (and often boring) speaker. This isn’t always the case and there several outlying examples but the general mean is below mediocrity.
IMO, conferences should try to limit the number of newbie talks to those present really ground breaking information. Yes, this makes it difficult for people to ‘break into’ the conference scene but the result is better information presented to the attendees. And isn’t that the primary reason for a conference? To present information to attendees? If you want to have a conference devoted to creating better speakers, fine, just don’t expect me to waste my time and money attending.
You are absolutely right. One of the goals of BSides is to give new speakers a chance where the audience is smaller, local, and more familiar. Using a BSides conference as a ‘proving grounds’ (the name of a BSidesLV track) is a great idea. Very few people are born with solid presentation skills. It is like any other skill we have, it takes practice. The most bad ass exploit developer may be the worst presenter, but mistakenly assume that one skill set applies to another.
How about a mini social session with the speaker after the talk? Say about 15 minutes. What I’ve seen at conferences quite a lot is audience members rushing up to talk to the speaker, perhaps exchange business cars, network, etc. Usually at this point the next speaker is trying to set up and get the next talk going. I think it would be useful to have built-in social breaks between talks. To me, this is usually the best part of the conference 🙂
My vision is to see an entire security conference (or a full track) dedicated to the security newb. From my perspective, the talks are all about the “rockstar” or some obscure edge case or 0-day which furthers a chasm between those that are there to learn, and those that are there to socialize. I think we sorely are missing a back to basics approach, and the fact that we continue to get popped by the basics only solidifies this argument.
Many of these are great ideas. I agree that there are too many cons. I will disagree with you on one thing, Heckling. No. It’s not a comedy club. I can’t think of anything more unprofessional then heckling. Q&A after the talk, yes. In the middle of the talk no.
Keep in mind that I use the word “heckling” to cover challenging speakers on points, calling them out if they are wrong, etc. I do not use it in the typical comedy club fashion.
In my opinion, infosec needs to cover ground than it currently does not cover. Including:
(1) education. Our current economy treats computers as a “magical” thing that does stuff. This imposes heavy responsibilities on a narrow slice of society. We do not need perfection, here, but we I think we need computer designs which expose their relevant abstractions when needed. Here, “education” is not so much classroom activities – for now we probably need useful error messages, informative failure modes and searchable labels. In the future, we probably need some respect for abstractions such as processes and files.
(2) specializations. Socially speaking, we need specialists or we cannot be efficient enough to solve useful problems. However single specialization is a social dead end – the resulting social structures fail too rapidly. We need dual specializations for our social systems to have integrity. Modern architectures involve deeply nested stacks of abstractions, and we do not want them all collapsing all at once.
(3) a sense of fun. Or, more specifically, social mechanisms which tolerate openness and avoid harming others. It’s great to preach that security and obscurity are different concepts, but the current focus on secrecy as the goal of security shows that no one really gets it.
Those baby step speakers you’re concerned about? You’ve got an entire audience of experts sitting there with some fledgling wannabe talking, and you do not take this as an opportunity for education?
Ok, yes, sure, as a mechanism for conveying new information from the speaker to the audience it’s a failure. And, hypothetically speaking, it’s a failure on the part of the conference organizers, on the part of the participants, and on the part of the marketers. But if we cannot learn from our mistakes, while they are happening, we don’t have a viable system.
My current attitude is: use these incidents as an opportunity to teach the presenters about the subject they are presenting. And do it gently enough that they get it.
Sure, you’ve burnt your 15 minutes or half an hour multiplied by the size of the audience. But if you think of this as an opportunity to teach the system what it needs to learn? That turns it into a different, and much harder problem.
But that’s the kind of problem real infosec needs to address.
I imagine some of you reading my above post are going “what the fuck is this idiot going on about” while others are maybe kind of nodding. I wrote it too hastily.
So… “than” in the first sentence should be “that”.
“respect for abstractions such as processes and files” was meant to be at the computer human interaction level – I’m talking about grandma here. If she doesn’t understand how to make her computer system secure then the computer network she participates in is not secure.
My “specializations” was mostly written with me focussed on quantum and electrical abstractions and the sorts of insecurities which can creep in at the manufacturing level. Or have some of you just been trusting that that stuff all magically “works”?
Put differently, computer security needs to start from the point of view that the system is already compromised – the question is how do we make it more secure?
And the answer is that we can do quite a lot. These things and problems can be observed in a variety of ways and isolating the problems is the first step to fixing them.
Computer security, in my mind, is “the person responsible for the device understands what the device is doing” and that’s an incremental process.
Love the post. I’m sharing it with the LASCON (Lonestar Application Security Conference) planning team. Thanks!
Since I’m the dork who does law talkin’ talks, what sort of material/presentations would be useful to the community?