Transparency and Openness in Government and Public Policy

By Paul Francis | April 10, 2024

Although most of my Commentaries discuss esoteric health policy issues, this Commentary is more of a rumination about the challenges of creating not only transparency but “openness” in New York State government and public policy in general.

To be clear, although I am critiquing some recent examples of lack of openness in New York State government, my intention is not to pick on Gov. Kathy Hochul or her administration. I could say many of the same things regarding openness and transparency about the three previous New York State administrations in which I served. But I want to use recent examples in making my point and the Hochul administration is in office now.

In her first speech after assuming office in August 2021, Gov. Hochul said she was committed to a “new era of transparency” in New York State government.[1] To her credit, she then directed every New York State agency and authority to develop a plan to increase transparency and accountability.[2] The “Transparency Reports” of all state agencies that were submitted in 2021 remain posted on the governor’s website and have been updated in 2024.[3]

It’s fair to say that New York State agencies expose a great deal of data to the public and have made more data available following Gov. Hochul’s 2021 directive. So why do so many people who work with the State think it is still hard to access data collected by the State and why do so many people still think of State government as being opaque?[4]

I think part of the answer is that “transparency” is too narrow a term. “Transparency” has become a set of rules about the disclosure of data and information.[5] Although there are still datasets that could be made publicly available and information based on those datasets that could be made more accessible, what really is missing in New York State government – and in fairness, governments generally – is not transparency of data, but “openness” in the way government operates and makes decisions. While transparency is a set of rules, openness is essentially a matter of culture. In our paper titled Democratization of Health Data, Information and Policy Analysis, we noted the perhaps apocryphal quote of Mikhail Gorbachev that, “before you can have perestroika, you must have glasnost.” In other words, before you can have effective restructuring (perestroika), you must have openness and transparency (glasnost).

Before talking specifically about transparency and openness as these concepts relate to New York State government, I want to describe two examples of openness from the private sector. The world’s largest hedge fund, Bridgewater Capital, famously has a policy of “radical transparency,”[6] in which virtually all aspects of decision-making and communication within the firm are transparent to all employees. It is difficult to summarize in a short paragraph what this means in practice (think recording of meetings and making most emails accessible to all employees), but in abstract terms, it’s about creating an idea meritocracy that values finding “truths” – as Bridgewater’s founder Ray Diallo calls them – by encouraging the most open exchange of views possible irrespective of an employee’s rank and with, for lack of a better term, as much symmetry of information as possible.

The second example from the private sector involves my experience working at Bloomberg LP for 20 months between 2008 and 2010, before returning to New York State government and politics. Bloomberg LP’s offices are set up as open spaces that resemble a Wall Street trading floor. There are no private offices or cubicles, just a connected series of 4-foot trading desks.[7] All conference rooms are surrounded by glass. Mike Bloomberg believes that open space environments foster transparency, collaboration, and communication among employees. Mayor Bloomberg brought this philosophy with him to government when he put himself and all the senior officials in his administration into a newly created Bullpen at City Hall.

The open office environment at Bloomberg is a little hard to get used to, but I found that it reduced gossip and paranoia about what was happening behind closed doors while fostering camaraderie and esprit de corps. The New York State Capitol is an architectural masterpiece, but its offices are a mixture of mausoleum grandeur and rabbit warren improvisation that do not help foster a culture of openness.

In thinking about how New York State government could develop a culture of openness, it’s helpful to identify the impulses that pull government in the other direction.

Government needs privacy in various stages of the development of policy. Officials need the freedom to entertain the possibility of policies they know will be controversial without fear of being attacked for even considering such ideas. Even when the outlines of a policy direction are settled upon, premature disclosure increases the risk that the proposal will be killed in the crib before there is an opportunity to present the idea in the best possible light and mitigate criticism by “socializing” the idea with critics and supporters. This need for privacy in policy deliberation is one reason why administrations are so averse to press leaks and why intragovernmental deliberations are generally shielded from freedom of information law (FOIL) requests.

Once a policy idea reaches the stage of being proposed, typically in the Executive Budget but occasionally through what is known as a “program bill,” the principle of secrecy in negotiations is considered sacrosanct. For a Budget to be enacted, the principals of the Executive, Senate, and Assembly need to reach three-way agreement on all issues, which inevitably involves trade-offs. Disclosure of a compromise on a particular issue will invite blowback from opponents on that issue, which not only threatens that particular compromise but could derail a larger package of potential agreements.

Nevertheless, this legitimate need for privacy in policy development and negotiation contributes to a culture in State government that is anything but open, irrespective of particular rules that require transparency. This lack of openness is perhaps the most pronounced in the Budget process – for good reasons and bad – but here are some other less obvious examples of how the lack of a culture of openness has a corrosive effect on New York State government.

• Large organizations tend to be insular in their thinking, and New York State government is no exception. My experience is that the State develops its policy agenda without a great deal of observation about what is going on in other states. When Jason Helgerson was appointed the Medicaid director in 2011, there was a fair amount of skepticism that someone from Wisconsin could teach New Yorkers anything about Medicaid. Jason brought with him from Wisconsin the concept of a Medicaid Global Cap and spearheaded the development of a strategy that, despite course corrections, still guides much of New York’s health policy. Although there is a lot of focus now on the California MCO tax, it turns out that Illinois in 2020 pulled off the same heist of having commercial plans pay only 1% of the MCO tax, with 99% of the net proceeds effectively coming from the federal government.8 The Illinois MCO tax happened on my watch, so when I say “God, I wish I thought of that,” I really mean it. Of course, policymakers in New York are generally aware of what other states are doing, but there is no institutionalized structure for going to school on other states’ policies and practices. As I have said in another context, “good artists borrow, great artists steal.”
• One reflection of the lack of a culture of openness is that State agencies generally don’t publicly disclose quantitative measures of performance. New York City has had for decades a performance management system structured around the “Mayor’s Management Report”[9] that publishes metrics of operational execution by City agencies. A number of other states, including Maryland and Washington, have well-developed performance management systems that articulate strategy goals, define the terms of success, and create accountability by measuring progress. In my first job in the Andrew Cuomo administration as Director of Agency Redesign, we tried to create a performance management system for New York, but we were never able to overcome cultural resistance to the idea within the agencies. Part of the problem with the performance management initiative was getting agreement about what should be measured – the ultimate outcome being sought or the accomplishment of “intermediate outcomes” over which the agencies have more direct control. My team thought that the crime rate should be a measure of the success of the Department of Criminal Justice Services (DCJS). DCJS thought a more appropriate measure would be “fingerprint processing cycle time.” This difference in perspective was the perfect metaphor for the larger problem.
• Another barrier to openness is that the culture of Communications staff from the governor’s office down to the agencies is to spin responses rather than just answering the question. In my experience, even the most straightforward questions from the press typically receive responses from Communications staff that spin the message, even when providing a straightforward answer to the question would not be damaging. In isolation, failure to answer the question that was asked is not fatal, but collectively, it contributes to the sense that the government believes, to borrow a phrase from Jack Nicholson in A Few Good Men, that the public “can’t handle the truth.”
• Another example: the Office of the Inspector General routinely does not provide the subjects of an investigation with any information about the specific charges that are being investigated, the source of the allegations (which could be described by the nature of the source rather than disclosure of the name of a specific source), or a description of how and when the investigation will be concluded. This is not an issue of public transparency, but it is a reflection of a mentality that is the antithesis of openness.
• A microcosm of the lack of openness in State government can be found in various aspects of a program called the Consumer Directed Personal Assistance Program (CDPAP). Although little-known outside of the world of healthcare policy, spending on CDPAP is the biggest fiscal problem in the Health budget that the administration is trying to address in the FY 25 Budget. Yet the State nowhere publishes the number of individuals who receive services through CDPAP or other information about the program. Enrollment in CDPAP began to surge in 2017, but it was not until late 2019 that senior policymakers recognized what was happening and began to address the issue. Part of the reason for that is that no data regarding CDPAP was readily available, so the problem remained out of sight. The damage of the lack of openness in government is not limited to the public: in the absence of robust data and information being reported out, policymakers themselves frequently don’t know what’s going on. In the case of CDPAP, the State was not trying to hide the information. But the State lacked a data and information infrastructure that would have identified that this was an important statistic to track.[10]

Without sacrificing the government’s need for privacy in the development and negotiation of policy and the deliberation of decision-making, how can the State create a culture of openness that addresses some of the drawbacks described above?

Because openness is more about a cultural change than a new set of rules, the concept doesn’t lend itself to a laundry list of prescriptive measures. That said, here are a few suggestions I think the Hochul administration should consider:

1. Create a dedicated unit that would have as its mission increasing the transparency of data, the accessibility of useful information, and contributing to a culture of openness. This is essentially what we suggested in our proposal[11] to create an Office of Health Data, Information, and Policy Analysis. The transparency charge directive to the Office should be to replicate the useful data and information made available in other states such as Massachusetts. The cultural charge should be to think creatively about how greater information-sharing – both with the public and across provider and governmental entities – could improve health (broadly defined to include behavioral health and what are now called “health-related social needs”).
2. Design and implement on a rolling basis by agency a performance management system for New York State government. Perhaps take a few areas such as Health, which could leverage the Office of Health Data described above, where the leadership is sympathetic to the idea and committed to the vision that improving public understanding of their priorities and programmatic progress would advance the mission of those agencies.
3. Foster openness to ideas from outside the usual suspects. Require your executive chamber leadership (including program staff, DOB unit chiefs, and counsel’s office) to get out in the field at least two days a month. The gravitational pull of the central office (whether in Albany or New York City) is enormously strong. To reach what the venture capitalists call “escape velocity”[12] from the gravitational pull of the status quo, you will need to push people out of the nest, inconvenient as it might be to their day-to-day work.
4. You won’t be able to break down agency silos, but you can perhaps bend them by creating a physical environment that increases interaction. It probably isn’t possible to create a permanent Bullpen in the Capitol in Albany or at the governor’s New York City office at 633 3rd Ave., but the Blue Room in Albany[13] is probably big enough to host a Bullpen one or two days a month (one or two days a quarter?) in which you could take advantage of the physical environment of proximity to encourage different portfolios to collaborate more and resolve inter-portfolio disagreements.
5. Seek to counteract the insular nature of New York State government by creating a few “Tiger Teams” to benchmark what other states (or even other countries) are doing to address problems in New York that are most likely national if not global in nature.
6. Replace the spin cycle in which everything is fabulous and moving in the right direction with a self-critical approach that recognizes the necessity of continuous improvement. Gov. Hochul’s recent acknowledgment of the dysfunction of the Office of Cannabis Management was a good example of this, but what made her statement notable was that having a government official acknowledge problems is such an exception to the general rule.
7. There are so many talented and dedicated people working in State agencies, but they are handicapped by a sclerotic civil service system and a hierarchical management structure that successful businesses abandoned decades ago. Let them out of their bureaucratic constraints and they will surprise you with how much better they can operate and how many good ideas for incremental improvement they have been carrying around.
8. Ask Mike Bloomberg for advice about how he created a more open culture in business and New York City government and how some of those approaches could be adapted for New York State.

* * *

After watching New York State government pretty closely for 20 years, my conviction is that the absence of transparency and openness often seems like a tactical success in the short run but proves to be a strategic failure in the long run. Certainly, I wish I had done more when I was in government to take steps that would have increased openness – at least in less controversial areas such as the reporting of health data and information. When I reflect on all the things I wish I had accomplished when I was in government, I sometimes think I should rename these Commentaries as “Confessions.”

But with the perspective of distance, it is more obvious to me than ever that New York State government needs a lot of perestroika to meet the challenges of the many seemingly intractable problems it is facing. Glasnost, in the form of a culture of openness, can be the first – and perhaps indispensable – step on that journey.

Paul Francis is the Chairman of the Step Two Policy Project. He served as the director of the Budget in 2007 and as the Deputy Secretary for Health and Human Services from 2015 – 2020, among other government positions. This note and other writings by the Step Two Policy Project are published on Substack and on the Step Two Policy website.

______

[1] https://edition.cnn.com/2021/08/24/politics/kathy-hochul-new-york-governor-speech/index.html 
[2] https://www.governor.ny.gov/sites/default/files/2021-12/Executive_Chamber_Transparency_Plan.pdf
[3] https://www.governor.ny.gov/hochul-administration-transparency-plans
[4] See, e.g., recommendations of the Citizens Budget Commission in its “NYS Budget Outlook”, March 7, 2024: “Four simple improvements should be made to bolster Medicaid data transparency, which can then be used by the civic community to recommend further program improvements.” Https://cbcny.org/research/nys-budget-outlook
[5] Indeed, privacy watchdogs have recently highlighted that commercial businesses engaged in data mining are the primary beneficiaries of the transparency efforts of State governments. https://www.nytimes.com/2024/03/11/nyregion/nj-opra-bill-public-records.html
[6] https://www.principles.com/principles/f6412dca-b3f9-4dd0-bb65-274869dd21ed
[7] The story goes that when Bloomberg returned to the company as CEO after his mayoralty was concluded, he found that a handful of top executives had acquired six-foot desks, which were normally reserved for software engineers. The story goes that Bloomberg told one of these top executives that the problem with a top executive having a six-foot desk is that it would make it just that much less likely that a first-year sales associate would approach the executive and offer his or her observations about what the company should be doing. The rogue six-foot desks were swapped out for four-foot desks the next weekend.
[8] “Of the total MCO tax of $1.22 billion, only an estimated $26.3 million will be paid by non-Medicaid MCOs.” https://www.civicfed.org/iifs/blog/new-illinois-health-insurer-tax-generate-11-billion-federal-revenues
[9] https://www.nyc.gov/site/operations/performance/mmr.page
[10] The raw data is reported to the State through publicly available institutional cost reports (ICRs), but the information is only accessible in a meaningful way if you have enough technical tools and sophistication to roll up the numbers from the ICRs.
[11] https://22bd584a-fab4-4177-ba23-ab0417da452c.usrfiles.com/ugd/22bd58_e99f9fccc4c74a5d9cb116c233f6c6d8.pdf
[12] https://www.linkedin.com/pulse/escape-velocity-business-your-launchpad-entrepreneurial-i44qc/
[13] The Blue Room is the second largest open space in the Capitol, second only to the ceremonial Red Room that is adjacent to the governor’s office. If the administration needed to use the Red Room as well to create a temporary Bullpen, I think even the ghost of Teddy Roosevelt would forgive it.